Software Engineer III
Electronic Arts3 days ago
Vancouver, CanadaSenior
Responsibilities
- Build automated testing frameworks, validation checks, and CI/CD pipeline integrations with reproducible diagnostics.
- Develop methods to explore complex software behavior, identify coverage gaps, and detect defects.
- Build automated checks that validate software behavior and data against requirements.
- Develop evaluation frameworks for generative AI and LLM-based systems using LLM-as-a-judge patterns, rubric-based scoring, and automated calibration.
- Apply generative AI, NLP, machine learning, heuristic, deterministic, and model-based approaches to software testing problems.
- Integrate testing tools into development pipelines, support delivered software, and produce reproducible diagnostics.
- Prototype machine learning and LLM capabilities and collaborate with engineering, design, and quality partners on requirements, code reviews, and blockers.
Requirements
- 5+ years of professional software engineering experience, including 2+ years of hands-on experience building, integrating, or evaluating AI/ML and LLM-based systems.
- Master of Science in Computer Science, Mathematics, or a related field, or equivalent professional engineering experience.
- In-depth experience with LLMs, generative AI, and NLP concepts, architectures, and failure modes.
- Experience developing generative-system evaluation frameworks, including dataset curation, model-as-a-judge design, validation criteria, error analysis, and quantitative and qualitative metrics.
- Proficiency in Python, algorithms, data structures, modular design, automated testing, debugging, and version control.
- Experience preparing datasets, defining validation criteria, and performing error analysis using quantitative ML metrics.
- Experience with NLP or large language models, including prompting, fine-tuning, or retrieval-augmented generation.
Benefits
- British Columbia compensation range of 122,300–170,700 CAD, depending on location.
- Three weeks of vacation per year to start and 10 days of sick time annually.
- Paid top-up to EI/QPIP benefits up to 100% of base salary for eligible maternity, parental, and adoption leave.
- Extended health, dental, and vision coverage; life insurance; disability insurance; and a retirement plan for regular full-time employees.
- Certain roles may be eligible for bonus and other incentive programs.
- Additional benefits may include healthcare coverage, mental well-being support, retirement savings, paid time off, family leaves, and complimentary games.
Tech Stack
Categories
About Electronic Arts
Electronic Arts creates next-level entertainment experiences that inspire players and fans around the world. Here, everyone is part of the story. Part of a community that connects across the globe. A team where creativity thrives, new perspectives are invited, and ideas matter. Regardless of your role, team, or location, this is a place where everyone makes play happen. Join us.