AI Benchmark Engineer | Native Language Specialist - Chinese Mandarin - Remote
LILT (Production)5 months ago
Remote, Taiwan or Remote, ChinaSenior
Responsibilities
- Design and engineer multilingual Terminal-Bench tasks for evaluating coding agents.
- Build realistic Chinese-language datasets, files, prompts, and task environments without relying on English translations.
- Develop reference implementations and highly reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, review, calibration review, audit, and automated quality-control processes.
- Identify multilingual software failure points involving encoding, Unicode, locale behavior, text processing, and terminal workflows.
Requirements
- 5+ years of industry experience in software engineering.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Chinese Mandarin fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive Terminal/CLI-based development experience and familiarity with coding agents.
- Deep technical understanding of encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe string operations.
- For applicable languages, familiarity with bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
- An updated CV in English and successful completion of a GenAI assessment are required to join the expert community.
Benefits
- Remote freelance work as an independent 1099 contractor with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects and a global expert community.
- Work availability fluctuates with project demand, and most accepted tasks require at least 2 hours per day or 10 hours per week.
- Contractors do not receive health insurance, paid time off, retirement contributions, or guaranteed hours and are responsible for their own taxes.
- Geographic restrictions may apply in regions subject to international embargoes or sanctions.