Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
6 days ago
Remote, Portugal +2 moreSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, conversations, and development history.
  • Design coding-agent tasks, prompts, and solution criteria from intermediate environment states.
  • Write functional and integration tests that accept valid solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least 5 years of software development experience.
  • Experience with Python and FastAPI, JavaScript/TypeScript and React, Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • Master’s degree in Computer Science, Software Engineering, Data Science/Data Analytics, Artificial Intelligence/Machine Learning, Computational Linguistics/Natural Language Processing, Information Systems, or a related field; a bachelor’s degree is accepted with 5 years of experience.
  • At least 3 years of professional experience in related roles or domains for QA-automation/testing or cybersecurity roles.
  • English proficiency at B2 level or higher and an English-language CV.

Benefits

  • Remote, part-time, freelance, project-based work that can fit around other professional or academic commitments.
  • Paid per accepted task, with rates based on qualification tier and efficiency and compensation up to the equivalent of $40 per hour.
  • Opportunity to work on advanced AI projects, build portfolio experience, and influence future AI systems.
  • Participation is project-based and not permanent employment.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me