AI Benchmark Engineer | Native Language Specialist - German - Remote
LILT (Production)8 months ago
Remote, GermanyEntry Level
Responsibilities
- Design and engineer high-signal Terminal-Bench tasks for evaluating coding agents in multilingual software scenarios.
- Build realistic task environments using datasets and files in the target language.
- Create native-language prompts and identify failure points in AI systems.
- Develop reference implementations and deterministic verifier scripts.
- Run benchmark configurations across Haiku, Sonnet, and Opus model tiers and calibrate task difficulty from Easy to Very Hard.
- Analyze execution logs and participate in creation, review, calibration, and audit quality-control processes.
- Evaluate multilingual handling of encoding, Unicode, locale conventions, text I/O, toolchain interoperability, and safe string operations.
Requirements
- At least 1 year of industry experience in software engineering or prompt engineering.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native fluency in German with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
- Deep technical understanding of multilingual text-processing issues, including encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, and interoperability.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in interfaces or artifacts.
Benefits
- Remote, independent-contractor work with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects and a global expert community.
- Project availability may fluctuate, and hours are not guaranteed; most accepted tasks require at least 2 hours per day or 10 hours per week.
- This is a 1099 contractor role without health insurance, paid time off, or retirement contributions, and contractors are responsible for their own taxes.
- Geographic restrictions may apply in regions subject to international embargoes or sanctions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.