AI Benchmark Engineer | Native Language Specialist - Korean - Remote

6 months ago

Remote, Korea, SouthSenior

Responsibilities

Design and engineer high-signal Terminal-Bench tasks for evaluating coding agents in multilingual software environments.
Create realistic Korean-language datasets, files, prompts, and task environments that preserve the target language.
Develop reference implementations and deterministic verifier scripts, using rubric-based judging only when necessary.
Run tasks against Haiku, Sonnet, and Opus, analyze execution logs, and calibrate difficulty from Easy to Very Hard.
Participate in creation, review, calibration review, audit, and automated quality-control processes to ensure benchmark fairness and integrity.

5+ years of industry software engineering experience.
A proven track record at leading technology companies and/or graduation from top-tier engineering universities.
Native or near-native Korean fluency with deep knowledge of grammar, register, and phrasing, plus high English proficiency.
Strong proficiency in Python, standard shell scripting, and data processing.
Extensive experience with Terminal/CLI-based development workflows and familiarity with coding agents.
Deep understanding of multilingual text-processing issues, including encoding and decoding, Unicode normalization, locale conventions, text I/O, toolchain interoperability, and safe string operations.

Remote, flexible independent-contractor work with control over schedule and workload.
Competitive rates and prompt payments.
Access to diverse AI and language-technology projects and a global expert community.
Work availability fluctuates with project demand; this is not intended as a full-time or primary-income role, and hours are not guaranteed.
Most accepted tasks require at least 2 hours per day or 10 hours per week.
As a 1099 contractor, the worker receives no health insurance, paid time off, or retirement contributions and is responsible for tax obligations.
Geographic restrictions may apply for regions subject to international embargoes or sanctions.

501-1,000 employees