AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote
LILT (Production)2 months ago
Remote, MexicoSenior
Responsibilities
- Design and engineer high-signal multilingual Terminal-Bench tasks for evaluating coding agents.
- Create realistic Spanish-language datasets, files, task environments, and prompts without relying on English translations.
- Develop reference implementations and reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, human review, calibration review, audit, and automated quality checks to ensure benchmark fairness and integrity.
Requirements
- At least 5 years of industry software engineering experience.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Spanish fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, data processing, and terminal/CLI-based development workflows.
- Working familiarity with coding agents and deep understanding of multilingual text-processing issues, including encoding, Unicode normalization, locale conventions, text I/O, toolchain interoperability, and safe string operations.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
Benefits
- Remote independent-contractor work with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language technology projects and a global expert community.
- No employee benefits such as health insurance, paid time off, or retirement contributions; hours are not guaranteed.
- Most accepted tasks require at least 2 hours per day or 10 hours per week.
- The role is a 1099 contractor position, and contractors are responsible for their own taxes; geographic restrictions may apply in embargoed or sanctioned regions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.