6 days ago
Responsibilities
- Design and build healthcare-domain evaluation systems for LLM capabilities and interpret and communicate their results.
- Build evaluation pipelines, curate and synthesize datasets, and develop automated analyses.
- Run experiments on prompting techniques using internal and industry benchmarks.
- Improve internal tooling for implementing, running, and analyzing evaluations.
- Design benchmarking and regression-testing capabilities for model accuracy, safety, and utility.
- Collaborate cross-functionally with research, product, design, and engineering teams and contribute to product roadmaps.
Requirements
- Bachelor’s or higher degree in computer science or a related technical discipline.
- Significant Python programming experience or machine learning research experience.
- Experience building with and around LLMs, including prompt, context, and harness engineering.
- Deep expertise evaluating and understanding agent performance.
- Experience collaborating across functions and delivering high-quality results amid ambiguity.
- Experience taking products or capabilities from zero to one with a bias toward shipping and learning.
- Ability to foster knowledge sharing and contribute to a positive, inclusive work environment.
- Preferred: healthcare technology or health-domain experience.
- Preferred: data engineering experience involving text dataset sourcing, curation, and processing at scale.
- Preferred: strong written and verbal communication skills and ability to work with product managers, designers, and engineers.
- Preferred: interest in learning emerging AI technologies, patterns, and best practices.
Benefits
- The role is based in the United Kingdom and is open for a minimum of five days, with applications accepted until the position is filled.
- Certain roles may be eligible for benefits and other compensation; additional benefits and pay information is provided through Microsoft’s corporate pay resources.
Tech Stack
Categories
About Microsoft
Every company has a mission. What's ours? To empower every person and every organization to achieve more. We believe technology can and should be a force for good and that meaningful innovation contributes to a brighter world in the future and today. Our culture doesn’t just encourage curiosity; it embraces it. Each day we make progress together by showing up as our authentic selves. We show up with a learn-it-all mentality. We show up cheering on others, knowing their success doesn't diminish our own. We show up every day open to learning our own biases, changing our behavior, and inviting in differences. Because impact matters. Microsoft operates in 190 countries and is made up of approximately 228,000 passionate employees worldwide.