Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
10 hours ago
Remote, India +6 moreSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, conversations, and development history.
  • Design challenging tasks from intermediate environment states and define clear criteria for successful solutions.
  • Write functional and integration tests that accept valid approaches and reject incorrect ones.
  • Review AI-agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least five years of software-development experience.
  • At least three years of professional experience in related QA automation/testing or cybersecurity roles.
  • Experience with Python and FastAPI, JavaScript/TypeScript and React, Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • A master's degree in a relevant field, or a bachelor's degree with five years of experience in the field.
  • English proficiency at B2 level or higher.
  • Applicants must submit a CV in English and indicate their English proficiency.

Benefits

  • Part-time, remote freelance work that can fit around primary professional or academic commitments.
  • Project-based participation rather than permanent employment.
  • Opportunity to work on advanced AI projects, build portfolio experience, and influence future AI systems.
  • Payment per accepted task, with an effective rate of up to $30 per hour equivalent.
Mindrift

About Mindrift

1,001-5,000 employees

Mindrift builds an expert-sourcing platform that connects domain specialists to project-based work training and evaluating generative AI models, including supervised fine-tuning, RLHF, evaluation, and red-teaming. It is built and operated by Toloka, part of Nebius Group, and run from Amsterdam, Netherlands. Work is fully remote and freelance, serving global technology companies developing and improving large AI systems.

Contact me