Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
6 days ago
Remote, RomaniaSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Create challenging coding-agent tasks from intermediate environment states.
  • Write prompts and define criteria for determining whether tasks are solved correctly.
  • Develop functional and integration tests that accept valid solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least 5 years of software development experience.
  • At least 3 years of professional experience in related QA-automation/testing or cybersecurity roles, as applicable.
  • Experience with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or above.
  • Master’s degree in a listed or related field, or a bachelor’s degree with 5 years of experience in the field.

Benefits

  • Part-time, remote, freelance, project-based work rather than permanent employment.
  • Flexible work that can fit around primary professional or academic commitments.
  • Paid per accepted task, with rates up to the equivalent of $50 per hour depending on qualification tier and efficiency.
  • Opportunity to work on advanced AI projects, enhance a portfolio, and influence how future AI models perform.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me