Sr. Staff Applied AI Engineer (Tech Lead)
OpenLoop Health8 hours ago
Remote, United States or Toronto, CanadaStaff+
Responsibilities
- Set the technical direction for agent runtime and orchestration, including tool use, context, handoffs, and framework decisions.
- Build AI evaluation systems, regression tests, and AI-graded evaluations to measure quality.
- Manage model access and operations, including provider routing, version upgrades, and provider-failure planning.
- Own AI observability, tracing, and cost attribution.
- Build retrieval and grounding systems over company data.
- Translate human workflows into agent workflows and determine which work requires human involvement.
- Protect patient data and address security and privacy across AI systems.
- Make cloud and data infrastructure decisions on GCP and partner with the Data Platform team.
- Conduct code and design reviews and mentor engineers new to AI.
- Explain AI trade-offs to product, operations, and clinical stakeholders.
Requirements
- 12 or more years of software engineering experience with technical leadership at tech lead, staff, or principal scope.
- At least two years of building and running LLM-based systems in production.
- Deep hands-on expertise in agent runtime or evaluation, with working knowledge across the broader AI engineering stack.
- Demonstrated experience measuring AI quality using real evaluation methods and data.
- Experience leading the technical direction of a small team from inception or near-inception.
- Preferred: healthcare or regulated-industry experience, including PHI or HIPAA.
- Preferred: GCP experience.
- Preferred: experience building an internal platform for other engineering teams.
- Preferred: classic machine learning experience alongside LLM experience.
- Preferred: experience scaling a team from one pod into multiple teams.
Benefits
- Medical, dental, and vision insurance.
- Flexible Spending Accounts and Health Savings Accounts.
- Generous paid time off.
- Hybrid-work flexibility.
- 401(k) with company match.
- Life insurance, pet insurance, and additional benefits.
Tech Stack
Categories
About OpenLoop Health
OpenLoop Health provides white-label telehealth infrastructure and clinical operations for digital health companies, health plans, and health systems. Its end-to-end services span provider staffing and licensing, payer coverage, a virtual care platform with AI, pharmacy and diagnostics coordination, RPM fulfillment, and revenue cycle management, enabling nationwide virtual care delivery. Founded in 2020 and headquartered in Des Moines, Iowa, the company is privately held and supports programs operating across all 50 U.S. states.