Mindrift

Senior AI Agent Evaluation Engineer

Mindrift
Apply
3 months ago
Remote, United KingdomSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Design tasks from intermediate environment states, including prompts and definitions of successful solutions.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least 5 years of software development experience.
  • At least 3 years of professional experience in related QA-automation/testing or cybersecurity roles or domains.
  • Experience with Python, FastAPI, JavaScript/TypeScript, React, Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • Master’s degree in a relevant field, or a bachelor’s degree with five years of experience in the field.
  • English proficiency at B2 level or higher and ability to submit a CV in English.

Benefits

  • Part-time, remote, freelance project-based work that can fit around other professional or academic commitments.
  • Opportunity to work on advanced AI projects, enhance a portfolio, and influence future AI systems.
  • Paid per accepted task, with rates depending on qualification tier and task efficiency; the posting states an equivalent rate of up to $50/hr.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me