Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
6 days ago
Remote, IrelandSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Design challenging developer tasks, prompts, and solution criteria for AI coding agents.
  • Write functional and integration tests that accept valid solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests using QA feedback.

Requirements

  • At least 5 years of software development experience.
  • Experience with Python, FastAPI, JavaScript, TypeScript, React, Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • B2+ English proficiency.
  • A master's degree in Computer Science, Software Engineering, Data Science or Data Analytics, Artificial Intelligence or Machine Learning, Computational Linguistics or NLP, Information Systems, or a related field; a bachelor's degree is accepted with 5 years of experience.
  • At least 3 years of professional experience in related QA-automation/testing or cybersecurity roles or domains.

Benefits

  • Remote, part-time, freelance project-based work.
  • Flexible participation that can fit around primary professional or academic commitments.
  • Opportunity to work on advanced AI projects and enhance a professional portfolio.
  • Paid per accepted task through project-based participation.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me