26 days ago
Edina, MN, USAStaff+
Base Salary
$135k - $237k/yr
Responsibilities
- Lead the end-to-end Quality Engineering and product validation strategy across data pipelines, ontologies, metrics, APIs, analytics, AI capabilities, and user-facing workflows.
- Design and implement automated validation frameworks for data ingestion, transformations, ontology mappings, business rules, metric calculations, APIs, application workflows, and AI-generated outputs.
- Establish data quality and reconciliation capabilities covering completeness, accuracy, consistency, source-to-target mappings, lineage, and transformation logic across healthcare data sources.
- Develop validation strategies for ontologies and operational metrics, including concepts, mappings, calculations, aggregations, and analytical outputs.
- Lead quality engineering for AI-enabled capabilities, including LLM evaluation, prompt and workflow testing, grounding and factuality validation, regression testing, agentic workflows, and human-in-the-loop evaluation.
- Build reusable test harnesses, representative datasets, automated regression suites, quality gates, and observability capabilities.
- Partner with Product, Software Engineering, Data Engineering, AI Engineering, and healthcare domain experts to define measurable acceptance criteria and resolve complex quality issues.
- Mentor engineers and advance practices for testability, automation, data quality, reliability, and AI quality while remaining hands-on with critical validation work.
Requirements
- At least 7 years of relevant experience is required.
- A relevant degree is preferred.
- Strong hands-on programming and automation skills using Python, Java, TypeScript/JavaScript, or similar languages, including experience building test frameworks, validation services, or engineering tooling.
- Experience validating complex data pipelines, ETL/ELT processes, transformations, analytical datasets, APIs, business rules, and calculated metrics, with strong SQL and data analysis skills.
- Experience designing automated data quality, reconciliation, source-to-target validation, regression testing, and end-to-end product validation approaches.
- Experience testing modern data-intensive and full-stack applications and integrating automated testing and quality gates into CI/CD pipelines.
- Experience evaluating AI-enabled applications using LLM evaluation, prompt testing, RAG or grounding validation, and agentic workflow testing.
- Experience with observability, telemetry, monitoring, cloud platforms, and modern DevSecOps practices.
- Palantir Foundry/AIP or comparable enterprise data and AI platform experience is preferred.
- Strong ownership, technical leadership, problem-solving, collaboration, and cross-functional influence skills.
Benefits
- Comprehensive benefits plan.
- The position is incentive eligible.
Tech Stack
Categories
