Mindrift

Freelance Agent Evaluation Engineer

Mindrift
Apply
6 days ago
Remote, Saudi ArabiaSenior

Responsibilities

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, conversations, and development history.
  • Design tasks from intermediate environment states, including prompts and definitions of what constitutes a correct solution.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.

Requirements

  • At least five years of software development experience.
  • Experience with Python and FastAPI, JavaScript and TypeScript, React, Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 or higher.
  • A master’s degree in computer science, software engineering, data science or analytics, artificial intelligence or machine learning, computational linguistics or NLP, information systems, or a related field.
  • A bachelor’s degree is accepted with five years of experience in the field.
  • At least three years of professional experience in related roles or domains, specifically for QA-automation/testing or cybersecurity roles.

Benefits

  • Remote, part-time freelance work that can fit around primary professional or academic commitments.
  • Project-based participation with opportunities to work on advanced AI projects and enhance a portfolio.
  • Opportunity to influence how future AI models understand and communicate in the field.
  • Payment per accepted task, with rates up to the equivalent of $50 per hour.
Mindrift

About Mindrift

1,001-5,000 employees
Contact me