6 days ago
San Francisco, CA, USA or New York, NY, USAMid Level / Senior
H1B Sponsor
Base Salary
$143k - $331k/yr
Responsibilities
- Build scalable infrastructure for ingesting, processing, curating, versioning, and serving first-party and third-party training data.
- Implement data provenance, lineage, licensing, consent, PII protection, access control, isolation, retention, deletion, geographic restrictions, approval, audit, and training-eligibility controls.
- Develop automated data acquisition and curation pipelines for classification, filtering, deduplication, quality scoring, semantic enrichment, and dataset construction.
- Build evaluation-to-data feedback loops using failure clustering, hard-example mining, long-tail discovery, capability-gap detection, and targeted dataset generation.
- Develop synthetic and AI-native data pipelines using LLMs, VLMs, and agents for data generation, labeling, filtering, validation, enrichment, and transformation.
- Create metrics and observability infrastructure for dataset quality, coverage, diversity, contamination, duplication, compliance, attribution, and model-capability impact.
Requirements
- A master's degree in computer science, mathematics, software engineering, computer engineering, or a related field plus 4+ years of relevant experience, or a bachelor's degree in one of these fields plus 6+ years of relevant experience, or equivalent experience.
- Software engineering experience using Python, SQL, and Spark, Flink, or Ray.
- Experience with AI training-data governance platforms, provenance, licensing and rights metadata, consent management, PII handling, policy enforcement, or auditable lineage is preferred.
- Experience managing third-party datasets, licensed content, data partnerships, or externally sourced data with contractual and usage restrictions is preferred.
- Experience building privacy- and security-aware systems involving first-party data, isolation, access controls, retention, deletion, or purpose limitation is preferred.
- Experience with data clean rooms, privacy-preserving processing, de-identification, confidential computing, or secure data collaboration is preferred.
- Experience building evaluation, failure-mining, data-generation, and training feedback loops is preferred.
- Experience with synthetic data, model graders, reward signals, hard-example mining, active learning, data-mixture optimization, or multimodal and agentic datasets is preferred.
- Understanding of modern LLM training workflows, including pre-training, SFT, RL or post-training, evaluation, and synthetic data, is preferred.
- Strong understanding of data governance, security, privacy, provenance, access control, and data lifecycle management is preferred.
Benefits
- Employees living within 50 miles of a designated U.S. Microsoft office or 25 miles of a designated non-U.S. office are expected to work onsite at least four days per week starting January 26, 2026, subject to local law and jurisdictional variation.
- Certain roles may be eligible for benefits and other compensation.
- The position will remain open for at least five days and accepts applications on an ongoing basis until filled.
Tech Stack
Categories
Data Engineering
About Microsoft
Every company has a mission. What's ours? To empower every person and every organization to achieve more. We believe technology can and should be a force for good and that meaningful innovation contributes to a brighter world in the future and today. Our culture doesn’t just encourage curiosity; it embraces it. Each day we make progress together by showing up as our authentic selves. We show up with a learn-it-all mentality. We show up cheering on others, knowing their success doesn't diminish our own. We show up every day open to learning our own biases, changing our behavior, and inviting in differences. Because impact matters. Microsoft operates in 190 countries and is made up of approximately 228,000 passionate employees worldwide.