AI Benchmark Engineer | Native Language Specialist - Chinese Mandarin - Remote
LILT (Production)6 months ago
Remote, Taiwan or Remote, ChinaSenior
Responsibilities
- Design and engineer multilingual Terminal-Bench tasks for evaluating coding agents.
- Build realistic Chinese-language datasets, files, prompts, and task environments without relying on English translations.
- Develop reference implementations and highly reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, review, calibration review, audit, and automated quality-control processes.
- Identify multilingual software failure points involving encoding, Unicode, locale behavior, text processing, and terminal workflows.
Requirements
- 5+ years of industry experience in software engineering.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Chinese Mandarin fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive Terminal/CLI-based development experience and familiarity with coding agents.
- Deep technical understanding of encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe string operations.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
- An updated CV in English and successful completion of a GenAI assessment are required to join the expert community.
Benefits
- Remote freelance work as an independent 1099 contractor with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects and a global expert community.
- Work availability fluctuates with project demand, and most accepted tasks require at least 2 hours per day or 10 hours per week.
- Contractors do not receive health insurance, paid time off, retirement contributions, or guaranteed hours and are responsible for their own taxes.
- Geographic restrictions may apply in regions subject to international embargoes or sanctions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.