Sr Lead Machine Learning Engineer
Upwork
about 2 months ago
Remote, Worldwide
Senior / Staff+
H1B Sponsor
Base Salary
$195k - $308k/yr
Responsibilities
- Design and implement comprehensive evaluation frameworks for agentic systems.
- Build benchmarking pipelines capturing nuanced success indicators.
- Lead development of observability tools for analyzing agent behavior.
- Translate qualitative and quantitative signals into actionable insights.
- Collaborate with researchers, engineers, and product teams on evaluation methodologies.
- Own benchmarking infrastructure for scalable evaluation across AI initiatives.
- Champion rigorous experimental design and statistical analysis.
Requirements
- Proven experience designing evaluation systems for agentic or LLM-based AI.
- Deep expertise in statistical experimentation and human-AI interaction assessment.
- Fluency in building data pipelines using Python, SQL, and distributed frameworks.
- Demonstrated ability to influence product and model roadmaps through insights.
- Adaptive-level proficiency in integrating AI tools into technical workflows.
Benefits
- Comprehensive medical coverage for you and your family.
- Unlimited PTO.
- 401(k) plan with matching.
- 12 weeks of paid parental leave.
- Employee Stock Purchase Plan.
Tech Stack
PythonSQL
Categories
AI & MLData Science