6 days ago
Remote, CanadaSenior
Responsibilities
- Build realistic virtual company environments with codebases, infrastructure, tickets, documentation, and conversations.
- Design tasks from intermediate environment states, including prompts and criteria defining successful solutions.
- Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
- Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
- Create evaluation datasets for AI coding agents rather than writing software from scratch or performing data labeling.
Requirements
- At least 5 years of software development experience.
- At least 3 years of professional experience in related QA automation/testing or cybersecurity roles, as applicable.
- Experience with Python and FastAPI, JavaScript/TypeScript and React, Docker, Postgres, Kafka, and Redis.
- Experience writing functional and integration tests.
- Master ’s degree in a relevant field, or a bachelor’s degree with five years of experience in the field.
- English proficiency at B2 level or higher.
- CV must be submitted in English with English proficiency indicated.
Benefits
- Part-time, remote, freelance project-based work that can fit around other professional or academic commitments.
- Work on advanced AI projects and build portfolio experience.
- Opportunity to influence how future AI models understand and communicate in the relevant field.
- Paid per accepted task, with an effective rate of up to $50 per hour equivalent depending on qualification tier and task efficiency.
- Participation is project-based and not permanent employment.
Tech Stack
Categories
