Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
6 days ago
Remote, Poland +4 moreSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Design tasks from intermediate environment states, including prompts and criteria defining successful solutions.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least 5 years of software development experience.
  • Experience with Python and FastAPI, JavaScript/TypeScript and React, Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • A Master's degree in a listed computing or related field, or a bachelor's degree with 5 years of field experience.
  • At least 3 years of professional experience in related roles or domains for QA automation/testing or cybersecurity roles.
  • English proficiency at B2 level or higher; CV must be submitted in English.

Benefits

  • Remote, part-time freelance work that can fit around primary professional or academic commitments.
  • Participation in advanced AI projects and portfolio-building opportunities.
  • Opportunity to influence how future AI models understand and communicate in the relevant field.
  • Project-based participation rather than permanent employment; work proceeds by applying, passing qualifications, joining a project, completing tasks, and getting paid.
  • Paid per accepted task, with compensation up to the equivalent of $40/hr depending on qualification tier and task efficiency.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me