AI Benchmark Engineer | Native Language Specialist - Japanese - Remote
LILT (Production)8 months ago
Remote, JapanEntry Level
Responsibilities
- Design and build high-signal Terminal-Bench tasks that evaluate coding agents in multilingual software environments.
- Create realistic Japanese-language datasets, files, prompts, and task assets without relying on English translations.
- Develop reference implementations and reliable deterministic verifier scripts.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in task creation, human review, calibration review, audit, and automated LLM-based quality checks.
- Ensure grammatical accuracy, fairness, benchmark integrity, and robust handling of encoding, Unicode, locale, and text-processing edge cases.
Requirements
- At least 1 year of experience in software engineering or prompt engineering.
- Proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Native or near-native Japanese fluency with deep knowledge of Japanese grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive Terminal/CLI-based development experience and familiarity with coding agents.
- Deep technical understanding of encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe string operations.
- For applicable languages, knowledge of bidirectional or RTL handling, font fallbacks, and rendering or typography in user interfaces or artifacts.
Benefits
- Remote, flexible independent-contractor work with no fixed hours or micromanagement.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects and a global expert community.
- 1099 contractor arrangement with no health insurance, paid time off, retirement contributions, or guaranteed hours.
- Most accepted tasks require at least 2 hours per day or 10 hours per week, and work availability fluctuates with project demand.
- Contractors are responsible for their own tax obligations, and geographic restrictions may apply in embargoed or sanctioned regions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.