2 days ago
Responsibilities
- Design and ship end-to-end features across data models, APIs, background workers, and UI.
- Build and improve agent and language-model evaluation workflows, including sandboxed trials, scoring, and result aggregation.
- Develop benchmark quality checks using verifiers, rubrics, and LLM-as-a-judge grading.
- Integrate models, agents, and benchmarks through generalized systems rather than one-off special cases.
- Improve query efficiency, migrations, concurrency, performance, and reliability under load.
- Review pull requests and maintain consistent engineering standards.
Requirements
- 5+ years of experience building and operating production backend systems, including ownership of at least one service from design through on-call.
- Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
- Strong Python, Django or similar framework, PostgreSQL, and task-queue experience such as Celery.
- Hands-on experience with LLM evaluation, including benchmark construction, agent-trial scoring, and result comparison.
- Ability to assess task clarity, verifier quality, leaked answers, and misleading checks.
- Comfort working with TypeScript and React when frontend changes are required.
- Clear technical writing and sound independent engineering judgment.
- Preferred experience includes Harbor, Terminal-Bench, SWE-bench, Inspect, sandboxed or containerized agents, Docker, Kubernetes, Modal, Daytona, distributed scheduling, coding agents, MCP, and open-source evaluation or infrastructure projects.
Benefits
- Work on frontier AI evaluation infrastructure for leading AI labs.
- Contribute to AI research and potentially showcase work at ICLR, ICML, and NeurIPS.
- Apply frontier AI innovation to enterprise workflows.
- Collaborate with colleagues with experience at Google, Meta, Amazon, and other technology companies.
- Startup-paced environment with significant ownership and impact.
- Equity is included with the compensation package.
Tech Stack
Categories
About Turing
Turing builds AI data pipelines and systems for enterprises and AI labs. It creates large-scale datasets and reinforcement-learning environments to train and evaluate models, and deploys agentic AI applications in domains like software engineering and enterprise workflows. The company also operates an AI-powered talent cloud to match and manage developers and AI trainers. Founded in 2018 and headquartered in San Francisco, it is privately held and serves Fortune 500 customers.