AI Benchmark Engineer | Native Language Specialist - Arabic - Remote
LILT (Production)6 months ago
Remote, EgyptSenior
Responsibilities
- Design and engineer Terminal-Bench tasks that evaluate coding agents on multilingual software challenges.
- Create realistic datasets, files, prompts, and task environments that remain in the target language.
- Develop reference implementations and deterministic verifier scripts for benchmark tasks.
- Run and calibrate tasks across Haiku, Sonnet, and Opus model tiers using standard Terminal-Bench configurations.
- Analyze execution logs and assess task difficulty from Easy to Very Hard.
- Participate in creation, review, calibration, and audit quality-control processes.
- Evaluate multilingual text handling, including encoding, Unicode normalization, locale conventions, bidirectional text, RTL behavior, font fallbacks, and rendering.
Requirements
- 5+ years of industry experience in software engineering.
- Native or near-native Arabic fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with terminal- or CLI-based development workflows and familiarity with coding agents.
- Deep technical understanding of multilingual text-processing issues, including encoding and decoding, Unicode normalization, locale-dependent conventions, text I/O, toolchain interoperability, and safe string operations.
- A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
- Reliable availability and commitment to complete accepted tasks on time; most tasks require at least 2 hours per day or 10 hours per week.
Benefits
- Remote freelance work as an independent 1099 contractor with flexible scheduling and no fixed hours.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects and a global expert community.
- No guaranteed hours; project availability fluctuates with demand.
- Contractors do not receive employee benefits such as health insurance, paid time off, or retirement contributions and are responsible for their own taxes.
- Geographic restrictions may apply in regions subject to international embargoes or sanctions.