1 day ago
Palo Alto, CA, USAMid Level
Base Salary
$200k - $300k/yr
Responsibilities
- Build autonomous AI agents that complete delegated work end to end with minimal user oversight.
- Define quality measures and evaluation frameworks using representative tasks, datasets, metrics, and simulations.
- Develop agent capabilities for existing tools and verify readiness before launch.
- Investigate production failures, improve agent recovery, and deliver durable measurable fixes.
- Build systems to detect known and unpredictable failure patterns at scale.
- Own systems through production launch and measure their impact.
Requirements
- At least 3 years of experience in data science or machine learning, including evaluation systems, metrics, or product-quality measurement.
- Strong production proficiency in Python and SQL, including large-dataset analysis and automated pipelines.
- Experience designing evaluation frameworks with measurable criteria, representative datasets, and appropriate metrics.
- Experience building or operating agents for real users and improving their reliability.
- Familiarity with statistics, experiment design, and realistic end-to-end simulations.
- Experience owning systems through production launch and measuring their impact.
- Experience with post-training, fine-tuning, or reinforcement learning is relevant.
- Working knowledge of Django, React, and TypeScript is useful.
- Typically fewer than 5 years of engineering experience with demonstrated progression or startup ownership.
Benefits
- Base salary of $200,000 to $300,000 annually.
- On-site in Palo Alto, California, with four days per week in the office.