AI Benchmark Engineer | Native Language Specialist - Chinese (Hong Kong) - Remote
LILT (Production)10 hours ago
Remote, Hong KongSenior
Responsibilities
- Design and engineer Terminal-Bench tasks that evaluate coding agents on multilingual software challenges.
- Build realistic datasets, files, and task environments in Chinese while preserving the target language.
- Identify AI failure points through native-language prompting and translation work.
- Develop reference implementations and reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus configurations.
- Participate in creation, human review, calibration review, audit, and automated quality-control processes.
- Ensure benchmark tasks address encoding, Unicode normalization, locale conventions, text I/O, toolchain interoperability, and safe string handling.
Requirements
- 5+ years of industry experience in software engineering.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native fluency in Chinese with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
- Deep technical understanding of multilingual text-processing pitfalls, including encoding and decoding, Unicode normalization, locale-dependent conventions, collation, casing, non-Gregorian dates, text I/O, and toolchain interoperability.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
Benefits
- Remote, flexible independent-contractor work with control over schedule and workload.
- Competitive rates and prompt payments.
- Access to diverse, innovative projects and a global community of linguists, subject-matter experts, and language professionals.
- Work availability fluctuates with project demand, making the role better suited to supplemental income than a primary full-time job.
- Most tasks require at least 2 hours per day or 15–20 hours per week, with no guaranteed hours.
- This is a 1099 contractor role without health insurance, paid time off, retirement contributions, or other employee benefits; contractors handle their own taxes.
- Geographic restrictions may apply for regions subject to international embargoes or sanctions.
- Application requires an English CV, a GenAI assessment, and onboarding/profile setup.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.