Principal AI/ML Engineer - Validation & Evaluation Governance
Mayo Clinic1 day ago
Rochester, MN, USAStaff+
Responsibilities
- Define enterprise AI/ML validation and evaluation standards, pathways, evidence requirements, and revalidation triggers.
- Review validation strategies, datasets, test plans, performance, safety, HCI, workflow, guardrails, production parity, subgroup evidence, and residual risk.
- Issue traceable review outcomes and direct corrections, additional evidence, alternate methods, interim controls, limitations, and escalations.
- Lead consultation and establish precedent for high-risk, novel, disputed, vendor-constrained, patient-facing, autonomous, and agentic cases.
- Develop policies, playbooks, rubrics, decision aids, evidence examples, case libraries, and escalation criteria.
- Lead training, calibration, office hours, case consultation, quality assurance, and coaching for AVM staff and product leads.
- Coordinate with clinical, technical, operational, governance, patient safety, legal, and executive stakeholders.
- Translate recurring gaps into improved guidance, vendor expectations, data needs, TRex workflows, evidence objects, dashboards, and technology priorities.
- Mentor junior engineers and provide technical leadership, with possible supervisory responsibilities.
Requirements
- Master’s degree in engineering, computer science, mathematics, health science, or a related field with 7 years of relevant experience, or a bachelor’s degree with 9 years of relevant experience.
- At least 7 years applying AI and machine learning in production healthcare or similarly regulated or technology-focused industries.
- Demonstrated leadership managing complex projects and leading technical or quantitative teams in regulated environments.
- Expertise in AI/ML techniques and frameworks, data engineering, data science, AI engineering, and MLOps practices.
- Experience with cloud infrastructure, software development tools, and large, complex, heterogeneous datasets.
- In-depth knowledge of healthcare technology, clinical workflows, electronic health records, medical terminologies, regulatory requirements, and industry standards.
- Experience creating risk management files and verification/validation strategies for digital health products.
- Expertise in user-centered design, human factors engineering, usability testing, and evaluation across AI product development.
- Experience mentoring and training team members and communicating complex technical findings to non-technical, clinical, operational, and executive audiences.
- Preferred: Ph.D. or other doctorate; healthcare informatics standards, common data models, regulatory and compliance frameworks, systems or quality engineering, standards organizations, and hands-on TRex assessment experience with AI tools deployed in Epic, ANIMATE, or comparable clinical environments.
Categories
About Mayo Clinic
Mayo Clinic is a nonprofit academic medical center and health system providing hospital and specialty care, research, and medical education to patients globally. It operates major campuses in Rochester, Minnesota (headquarters), Phoenix/Scottsdale, Arizona, and Jacksonville, Florida, plus a regional health system across the Midwest. Revenue comes primarily from clinical services, complemented by research grants and education programs; specialties include heart care, cancer, transplantation, and neurosciences.