Senior Machine Learning Engineer
Ambience HealthcareBase Salary
$225k - $300k/yr
Responsibilities
- Design and own evaluation pipelines for LLM and agentic systems using automated graders, regression testing, production feedback, and human evaluation.
- Diagnose production model failure modes and test improvements involving prompting, retrieval, context, routing, data, fine-tuning, and other model or system interventions.
- Develop production agentic AI systems with tool use, retrieval, context and state management, routing, orchestration, tracing, and failure recovery.
- Convert production failures and user feedback into improved datasets, evaluations, and model behavior through active learning and systematic iteration.
- Distill recent research in LLMs, agents, NLP, speech, and multimodal AI into practical experiments.
- Own AI systems end-to-end across models, data, evaluation, orchestration, serving, and observability while writing code and debugging production systems.
Requirements
- At least five years of experience in production machine learning, research engineering, or applied AI.
- Experience building consequential production AI systems or materially improving model behavior in production.
- Strong understanding of modern large language models, transformers, and production AI systems.
- Experience designing evaluations for LLMs, agents, or other complex AI systems, including datasets, measurable quality dimensions, regression detection, and offline-online analysis.
- Experience building production systems involving multiple models, tools, retrieval, context, state, routing, or orchestration.
- Proficiency in Python and modern machine learning frameworks; PyTorch is preferred.
- Comfort with deployment, observability, CI/CD, and containerized systems, along with hands-on production debugging.
- Skill in building high-quality datasets and feedback loops using production failures, user feedback, and active learning.
- Ability to collaborate with clinicians, product managers, and engineers and communicate complex AI concepts clearly.
- Nice-to-have experience with realtime voice, conversational AI, multimodal systems, fine-tuning, post-training, model adaptation, healthcare or regulated industries, interviewing or mentoring ML engineers, or open-source ML, agent, or evaluation tooling contributions.
Benefits
- Hybrid engineering role requiring onsite work at the San Francisco office three days per week, with a remote-friendly culture.
- Comprehensive medical, dental, and vision coverage for employees and dependents.
- 401(k) with company match of up to 3% of base salary.
- Full equipment provisioning.
- Parental leave.
- Annual company-wide and team off-sites, team lunches, and all-hands gatherings with travel, lodging, and meals covered.
- Flexible time off with no annual cap, company-wide holidays, and an annual holiday shutdown from December 24 through January 1.
About Ambience Healthcare
Ambience Healthcare is the leading AI platform for documentation, coding, and clinical workflows, built to reduce administrative burden and protect revenue integrity at the point of care. Trusted by top health systems across North America, Ambience’s platform is live across outpatient, emergency, and inpatient settings, supporting more than 100 specialties with real-time, coding-aware documentation. The platform integrates directly with Epic, Oracle Cerner, athenahealth, and other major EHRs. Founded in 2020 by Mike Ng and Nikhil Buduma, Ambience is headquartered in San Francisco and backed by Oak HC/FT, Andreessen Horowitz (a16z), OpenAI Startup Fund, Kleiner Perkins, and other leading investors. For official information or questions on available roles, please visit https://careers.ambiencehealthcare.com