AI Benchmark Engineer | Native Language Specialist - Portuguese (Brazil) - Remote
LILT (Production)21 hours ago
Remote, BrazilEntry Level
Responsibilities
- Design and engineer high-signal Terminal-Bench tasks for evaluating coding agents in multilingual software environments.
- Create realistic Portuguese-language task assets, datasets, files, prompts, and translations without relying on English translations.
- Develop reference implementations and reliable deterministic verifier scripts, using rubric-based judging only when necessary.
- Analyze execution logs and calibrate task difficulty from Easy to Very Hard across Haiku, Sonnet, and Opus model tiers.
- Participate in creation, human review, calibration review, audit, and automated quality-control processes.
- Identify and test multilingual text-processing issues involving encoding, Unicode normalization, locale conventions, text I/O, and toolchain interoperability.
Requirements
- At least 1 year of industry experience in software engineering or prompt engineering.
- Proven track record at leading technology companies and/or graduation from a top-tier engineering university.
- Native or near-native fluency in Portuguese (Brazil), deep knowledge of grammar, register, and phrasing, and high English proficiency.
- Strong proficiency in Python, standard shell scripting, and data processing.
- Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
- Deep technical understanding of encoding and decoding, Unicode normalization, locale-dependent collation and casing, non-Gregorian dates, text I/O, interoperability, and safe string operations.
- Updated CV in English and successful completion of a GenAI assessment are required during the application process.
Benefits
- Remote, flexible independent-contractor work with the ability to choose schedule and workload.
- Competitive rates and prompt payments.
- Access to diverse AI and language-technology projects that can expand the contractor’s portfolio and experience.
- Participation in a global community of linguists, subject-matter experts, and language professionals.
- Work availability fluctuates with project demand; hours are not guaranteed and the role is not intended as a primary full-time income source.
- Most accepted tasks require at least 2 hours per day or 10 hours per week.
- The role is a 1099 contractor position without health insurance, paid time off, retirement contributions, or other employee benefits.
- Contractors are responsible for their own taxes, and geographic restrictions may apply in regions subject to international embargoes or sanctions.
About LILT (Production)
LILT builds an enterprise translation and localization platform that combines AI models with human linguists to produce multilingual content for businesses and governments. The company also provides data labeling and evaluation services for AI developers, plus managed translation services integrated with common content and support tools. Founded in 2015 and headquartered in San Francisco, it is privately held and works with global organizations that need at-scale, domain-specific localization.