9 days ago
Base Salary
$143k - $304k/yr
Responsibilities
- Own CADET’s end-to-end architecture, integration contracts, and technical execution across customer-signal and evaluation workflows.
- Build scalable services and pipelines for grounding-data generation, classification, deduplication, clustering, taxonomy assignment, coverage analysis, loss detection, and work-item routing.
- Automate recurring evaluations and quality gates, including scheduling, versioning, reproducibility, failure handling, and result publication.
- Establish extensible data contracts and schemas for signals, workflows, evaluation cases, rubrics, graders, runs, results, attribution, and regression assets.
- Develop automated diagnosis and human-review workflows that route quality losses, track remediation, and verify customer-impact improvements.
- Create observability and benchmarks for pipeline health, agent behavior, data quality, grader health, coverage, cost, latency, reliability, and regression detection.
- Design safe human-in-the-loop gates for ambiguous data, sensitive evidence, taxonomy changes, evaluation publication, and consequential quality decisions.
- Mentor engineers and improve engineering quality through clear interfaces, testing, security, operational discipline, and technical documentation.
Requirements
- Bachelor’s degree in Computer Science or a related technical field and 6+ years of technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
- Extensive experience designing and delivering large-scale distributed systems, data platforms, developer platforms, or AI infrastructure in production is preferred.
- Experience with large language model technologies, agent systems, AI/ML platforms, evaluation infrastructure, experimentation systems, or model pipelines is preferred.
- Experience with retrieval, tool-using agents, prompt and rubric management, human-in-the-loop systems, or automated diagnosis is preferred.
- Experience building data and workflow platforms combining batch, streaming, unstructured, and human-generated inputs is preferred.
- Experience with RLEs, synthetic environments, post-training pipelines, or large-scale evaluation execution is preferred.
- Experience integrating customer feedback, product telemetry, and engineering work-management systems is preferred.
- Experience operating high-availability services and driving platform adoption across multiple product teams is preferred.
- Demonstrated principal-level technical leadership, cross-organizational influence, and knowledge of security, privacy, data governance, and responsible engineering are preferred.
Benefits
- The typical U.S. base pay range is USD $142,800-$274,800 per year; a separate San Francisco Bay Area and New York City metropolitan area range is USD $188,000-$304,200 per year.
- Certain roles may be eligible for benefits and other compensation.
- The position is open for a minimum of 5 days, with applications accepted on an ongoing basis until filled.
Tech Stack
Categories
BackendData Engineering
About Microsoft
Microsoft develops operating systems, productivity software, cloud services, developer tools, and consumer devices for individuals, enterprises, and governments. Its main products include Windows, Microsoft 365, Azure, Visual Studio/GitHub, Xbox, and LinkedIn; revenue comes from software subscriptions and licenses, cloud consumption, hardware sales, and advertising. Founded in 1975 and headquartered in Redmond, Washington, Microsoft is a public company traded on Nasdaq.