Sr Software Development Engineer in Test, ML
PlayStation Global5 months ago
Base Salary
$183k - $275k/yr
Responsibilities
- Design, develop, and maintain automated test frameworks and scripts for ML inference services and related ML platform components.
- Create realistic test accounts and datasets to evaluate model behavior across scenarios.
- Validate model score distributions, ranking behavior, output quality, and other signals before release.
- Perform functional, integration, regression, API, performance, and load testing for distributed services, databases, and inference systems.
- Collaborate with ML engineers and operations teams to provide automated test coverage for features and bug fixes.
- Debug test failures and model anomalies and report defects.
- Drive scalable, maintainable test automation and quality standards for engineering decisions and customer-facing features.
- Monitor test execution and optimize performance in test environments.
Requirements
- Bachelor's degree or equivalent experience.
- At least 5 years of experience as an SDET with strong backend systems testing experience.
- Proficiency in Python, Java, Scala, or Go for test automation.
- Hands-on experience with API testing tools such as Postman for distributed systems.
- Familiarity with CI/CD pipelines and environments such as Jenkins, GitHub Actions, and ArgoCD.
- Experience building and maintaining automated test frameworks such as pytest, JUnit, or TestNG.
- Experience working with databases for backend validation.
- Experience with cloud and container technologies such as AWS, GCP, Kubernetes, and Docker.
- Familiarity with monitoring and observability tools such as Prometheus, Grafana, and Datadog.
- Strong understanding of software development lifecycle and agile methodologies.
- Excellent problem-solving and communication skills.
- Preferred experience testing recommendation systems, search ranking systems, or other probabilistic ML applications.
- Preferred experience creating synthetic or seeded test data to simulate realistic customer and account behaviors.
- Preferred experience evaluating model outputs with statistical techniques, distribution analysis, or scenario-based validation.
- Knowledge of data pipelines, feature stores, inference systems, or model-serving infrastructure such as Seldon, KServe, and Ray Serve is a plus.
- Experience testing high-RPS, low-latency online services is preferred.
- Experience with performance and load testing tools such as Locust, k6, Gatling, or JMeter is preferred.
- Experience with Databricks or similar ML platform tooling is preferred.
Benefits
- Medical, dental, and vision coverage
- Matching 401(k)
- Paid time off
- Wellness program
- Employee discounts on Sony products
- Potential eligibility for a bonus package
- Hybrid working policy applies
- Background checks are conducted at the offer stage
Tech Stack
Apache JMeterAWSDatabricksDatadogDockerGitHub ActionsGoGoogle Cloud PlatformGrafanaJavaJenkinsJUnitKubernetesPostmanPrometheuspytestPythonScalaTestNG