Lead Software Development Engineer in Test - AI Tooling & Automation
Salesforce7 days ago
Base Salary
$173k - $286k/yr
Responsibilities
- Design, build, and maintain AI-first testing frameworks and automation suites for complex workflows and agentic capabilities.
- Translate quality strategies into automation plans for LLM integrations, prompt accuracy, and deterministic system behavior.
- Perform white-box testing, analyze application architecture, identify quality gaps, fix code, and address test flakiness.
- Implement human-in-the-loop evaluation processes for response relevance, user safety, and contextual accuracy.
- Collaborate with developers, product managers, AI/ML engineers, and data scientists to integrate quality throughout software delivery.
Requirements
- Strong Java skills, including data structures and algorithms.
- Hands-on experience evaluating large language model responses and familiarity with GPT-like models or AI evaluation frameworks is advantageous.
- Proven experience delivering automation solutions in fast-paced CI/CD environments.
- Strong coding background and experience delivering software at pace in an agentic product development lifecycle.
- Strong analytical, problem-solving, communication, and cross-functional collaboration skills.
- Experience with white-box testing and fixing application code is preferred.
- Experience with AWS, Azure, or GCP and foundational Salesforce platform knowledge are preferred.
- Experience testing autonomous agents, conversational user interfaces, or generative AI workflows is preferred.
Benefits
- Benefits include time off programs, medical, dental, vision, mental health support, paid parental leave, life and disability insurance, 401(k), and an employee stock purchasing program.
- The posting states that certain roles may be eligible for incentive compensation and equity, in addition to benefits.
Tech Stack
Categories
About Salesforce
We're the #1 AI CRM—where humans with agents drive customer success together with AI, data, and Customer 360 apps on one platform. Privacy Statement: http://www.salesforce.com/company/privacy/