6 days ago
Remote, EstoniaSenior
Responsibilities
- Build realistic simulated software-development environments containing codebases, infrastructure, tickets, documentation, and conversations.
- Design developer tasks from intermediate environment states and define clear, solvable success criteria for AI coding agents.
- Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
- Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
- Create challenging evaluations that reveal meaningful differences between strong and weak AI coding-agent solutions.
Requirements
- At least 5 years of software development experience.
- At least 3 years of professional experience in related QA automation/testing or cybersecurity roles or domains.
- Experience with Python, FastAPI, JavaScript/TypeScript, React, Docker, Postgres, Kafka, and Redis.
- Experience writing functional and integration tests.
- A master’s degree in computer science, software engineering, data science/data analytics, artificial intelligence/machine learning, computational linguistics/natural language processing, information systems, or a related field; a bachelor’s degree is accepted with five years of field experience.
- English proficiency at B2 level or higher.
- Submit a CV in English and indicate English proficiency.
Benefits
- Part-time, remote, freelance work that can fit around primary professional or academic commitments.
- Project-based participation rather than permanent employment.
- Work on advanced AI projects and build portfolio experience.
- Influence how future AI models understand and communicate in software-related fields.
- Paid per accepted task, with an effective rate of up to the equivalent of $40/hr depending on qualification tier and task efficiency.
Tech Stack
Categories
