AI Benchmark Engineer | Native Language Specialist - Dutch - Remote
LILT (Production)5 hours ago
Remote, NetherlandsSenior
Responsibilities
- Design and build high-signal Terminal-Bench tasks for evaluating coding agents in multilingual software scenarios.
- Create realistic Dutch-language datasets, files, prompts, and task environments that preserve the target language.
- Develop reference implementations and deterministic verifier scripts, using rubric-based judging only when necessary.
- Run benchmark tasks against Haiku, Sonnet, and Opus model tiers and calibrate difficulty from Easy to Very Hard.
- Analyze execution logs and identify failure points in multilingual AI behavior.
- Participate in creation, human review, calibration review, audit, and automated quality-control processes.
- Ensure grammatical accuracy, fairness, reproducibility, and benchmark integrity.
Requirements
- At least 5 years of industry software engineering experience.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Dutch fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
- Deep understanding of encoding and decoding robustness, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe string operations.
- For relevant languages and artifacts, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography.
Benefits
- Remote, flexible independent-contractor work with the ability to choose schedule and workload.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects that can expand professional experience and portfolio.
- Participation in a global community of linguists, subject-matter experts, and language professionals.
- This is a 1099 freelance opportunity with fluctuating project demand, no guaranteed hours, and no health insurance, paid time off, or retirement contributions.
- Most accepted tasks require at least 2 hours per day or 15–20 hours per week; geographic restrictions may apply, and contractors are responsible for their own taxes.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.