AI Benchmark Engineer | Native Language Specialist - German (Austria) - Remote
LILT (Production)2 months ago
Remote, AustriaSenior
Responsibilities
- Design and engineer Terminal-Bench tasks that evaluate coding agents on multilingual software challenges.
- Create realistic German-language datasets, files, prompts, and task environments without relying on English translations.
- Develop reference implementations and highly reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, human review, calibration review, audit, and automated quality-control processes.
- Apply expertise in encoding, Unicode normalization, locale conventions, text I/O, interoperability, and safe string operations.
Requirements
- At least 5 years of industry software engineering experience.
- Native or near-native German fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with terminal or CLI-based development workflows and familiarity with coding agents.
- Deep technical understanding of multilingual text-processing pitfalls, including encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, and toolchain interoperability.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
Benefits
- Remote, flexible independent-contractor work with control over schedule and workload.
- Competitive rates and prompt payments.
- Access to diverse applied-AI projects and a global community of linguists, subject-matter experts, and language professionals.
- This is a 1099 freelance opportunity with fluctuating availability, no guaranteed hours, and no health insurance, paid time off, or retirement contributions.
- Most accepted tasks require at least 2 hours per day or 10 hours per week.
- Contractors are responsible for their own taxes, and geographic restrictions may apply in regions subject to international embargoes or sanctions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.