Senior AI Reliability Engineer (Platform)
Flatiron Healthabout 5 hours ago
Responsibilities
- Design and improve evaluation frameworks and automated testing pipelines for AI workflows.
- Define and monitor quality, reliability, safety, performance, and cost metrics for AI systems.
- Develop reliability engineering practices for AI-enabled systems, including monitoring and incident response.
- Design orchestration and governance for multi-agent AI systems.
- Collaborate with various teams to evaluate AI solutions and establish reusable standards.
- Support experimentation with emerging AI technologies and participate in on-call rotations.
Requirements
- 5+ years of experience in platform engineering, SRE, machine learning, or related fields.
- Strong Python skills and experience building production-quality systems.
- Experience designing experiments and evaluation frameworks for ML or AI systems.
- Strong understanding of AI reliability and observability practices.
- Familiarity with modern cloud and ML infrastructure, including AWS and Kubernetes.
- Excellent communication skills to explain AI behavior to diverse stakeholders.