Senior Manager, AI Engineer/ Solutions & Engineering
Bristol Myers SquibbBase Salary
$154k - $186k/yr
Responsibilities
- Build and ship LLM-powered assistant extensions, connectors, agentic workflows, and custom applications from prototype to production.
- Define golden datasets, graders, human-review processes, regression suites, and CI-based evaluations for retrieval, tool use, agent trajectories, and final outputs.
- Use coding agents to write specifications, direct implementation, review generated code, and own correctness and quality.
- Design prompts, reusable instructions, memory, retrieval, structured outputs, and long-context strategies around AI assistant harnesses.
- Choose among platform extensions, connectors, custom applications, and API-based agents, and select models based on capability, cost, latency, and risk.
- Implement guardrails, human-in-the-loop controls, auditability, compliance, and data boundaries for regulated pharmaceutical content.
- Monitor and improve deployed solution quality, cost, latency, adoption, and usage through production traces and feedback.
- Partner with marketing, commercial, and IT stakeholders to translate business needs into technical solutions and enable responsible AI adoption.
- Share reusable engineering patterns, review peers' work, and track new models and platform capabilities.
Requirements
- Bachelor's degree in Computer Science, Engineering, Data Science, or a related field, or equivalent experience.
- At least 5 years of software or ML engineering experience, including at least 1 year building LLM or agent-based solutions.
- Strong Python skills, ideally with TypeScript, plus API-driven development, Git, CI/CD, and experience with AWS, Azure, or GCP.
- Proven experience delivering LLM solutions in production and a portfolio of shipped assistant extensions, connectors, agent workflows, or evaluation suites.
- Advanced prompt and context engineering, retrieval and grounding, agentic patterns, tool and function calling, MCP, and structured outputs.
- Knowledge of transformer fundamentals, model training concepts, agent harnesses, AI failure modes, and cost, latency, and quality trade-offs.
- Experience with evaluation and quality engineering, including LLM-as-judge graders, human calibration, regression testing, error analysis, and failure categorization.
- Understanding of LLMOps, tracing, observability, prompt and model versioning, cost and usage management, prompt injection, data leakage, PII, and least-privilege access.
- Preferred experience includes enterprise AI assistant platforms, MCP servers, agent SDKs, regulated industries, pharma systems, enterprise AI rollout, LangGraph, multimodal workloads, and fine-tuning or distillation with open-weight models.
- Strong communication skills and the ability to explain technical trade-offs to business partners.
Benefits
- Comprehensive benefits may include wellbeing support, retirement and financial protection benefits, and medical, dental, vision, life, and disability insurance.
- U.S.-based exempt employees receive Flexible Time Off without a set accrual limit, subject to manager approval, and 11 paid company holidays.
- Additional eligible benefits may include paid sick leave, up to two paid volunteer days, summer hours flexibility, various leaves of absence, and an annual Global Shutdown.
- The position is based in Princeton, New Jersey and is designated onsite per the #LI-Onsite posting marker.
Categories
About Bristol Myers Squibb
At Bristol Myers Squibb, we work every day to transform patients’ lives through science. That work inspires some of the most interesting, meaningful, and life-changing careers you’ll experience. Join us and pursue innovative ideas alongside some of the brightest minds in biopharma, collaborating with a team rich in diversity of experiences, and perspectives. We have built a sustainable pipeline of potential therapies and are leveraging translational medicine and data analytics to understand how we can deliver the right medicine to the right patient, at the right time, to achieve the best outcome. Whether in a scientific, business or supporting function, a career at BMS means you’ll be inspired every day to grow and thrive through opportunities that are uncommon in scale and scope. Here, you’ll be on the cutting edge of powerful innovation in oncology, hematology, immunology, cardiovascular disease, and fibrosis, with colleagues united in the mission to help patients. Through the Bristol Myers Squibb Foundation, we also promote health equity and seek to improve health outcomes of populations disproportionately affected by serious diseases and conditions. Our mission is to give new hope to help patients prevail over serious disease – it drives everything we do. Review our Social Media Community Guidelines at: https://www.bms.com/social-media-community-guidelines.html