AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote
LILT (Production)12 hours ago
Remote, MexicoSenior
Responsibilities
- Design and engineer high-signal multilingual Terminal-Bench tasks for evaluating coding agents.
- Create realistic Spanish-language datasets, files, task environments, and prompts without relying on English translations.
- Develop reference implementations and reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, human review, calibration review, audit, and automated quality checks to ensure benchmark fairness and integrity.
Requirements
- At least 5 years of industry software engineering experience.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Spanish fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, data processing, and terminal/CLI-based development workflows.
- Working familiarity with coding agents and deep understanding of multilingual text-processing issues, including encoding, Unicode normalization, locale conventions, text I/O, toolchain interoperability, and safe string operations.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
Benefits
- Remote independent-contractor work with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language technology projects and a global expert community.
- No employee benefits such as health insurance, paid time off, or retirement contributions; hours are not guaranteed.
- Most accepted tasks require at least 2 hours per day or 10 hours per week.
- The role is a 1099 contractor position, and contractors are responsible for their own taxes; geographic restrictions may apply in embargoed or sanctioned regions.