Software Development Engineer in Test - Agentic Automation (Remote, CAN)
CrowdStrike2 days ago
Remote, CanadaSenior
Responsibilities
- Design, develop, and maintain automated test suites and frameworks for distributed cloud-based systems.
- Extend agentic test-generation and analysis tooling within the testing frameworks.
- Build AI-powered workflows and agent skill files for test-data setup, failure triage, and release validation.
- Maintain eval suites, golden datasets, and scoring pipelines for AI agents.
- Enforce CI evaluation gates for changes to prompts, models, tools, and orchestration.
- Detect and mitigate unreliable AI output, including hallucinated assertions, flaky tests, incorrect failure triage, and judge bias.
- Design, run, and analyze performance, load, and stress tests to validate scalability and identify bottlenecks.
- Validate platform systems, services, data, applications, and customer-facing features end to end.
- Review product, system, and agent designs for testability, evaluability, flaws, and improvements.
- Contribute to team quality and evaluation standards and share knowledge.
Requirements
- Degree in Computer Science or equivalent experience, with knowledge of object-oriented programming, data structures, algorithms, and distributed systems.
- At least 5 years of experience in software quality, including at least 3 years of hands-on SDET or test-automation development.
- Strong programming skills in at least one of Java, Node.js, or Python; Python is preferred for evaluation and agent work.
- Experience designing, building, and maintaining test frameworks integrated with CI/CD pipelines such as Jenkins or GitLab CI/CD.
- Experience testing web services and APIs, including REST, gRPC, and JSON.
- Hands-on performance and load-testing experience using tools such as Locust or k6, including workload design and result interpretation.
- Ability to apply deterministic assertions and probabilistic evaluation appropriately when verifying AI output.
- Experience using AI technologies to improve decision-making, workflows, efficiency, and business outcomes.
- Preferred experience with eval-driven development, release gating, trajectory and tool-use evaluations, and calibrating LLM judges against human review.
- Preferred experience with prompt engineering, context or tool design for agents, CI/CD-integrated performance testing, and observability tools.
- Cybersecurity, threat intelligence, malware analysis, or sandboxing/detonation experience is a bonus.
Benefits
- Market-leading compensation and equity awards, plus variable or incentive compensation.
- Comprehensive physical and mental wellness programs.
- Competitive vacation and holidays.
- Paid parental and adoption leave.
- Professional development opportunities for all employees.
- Employee networks, geographic neighborhood groups, and volunteer opportunities.
- Vibrant office culture with world-class amenities.
- Remote Canadian-based employment; employees must maintain legal entitlement to work in Canada and may undergo background checks including criminal-record, credit, and reference checks.
Tech Stack
Categories
About CrowdStrike
CrowdStrike builds the Falcon cloud-native security platform used by enterprises and governments to protect endpoints, cloud workloads, identities, and data with EDR, next-generation antivirus, threat intelligence, and managed threat hunting. Founded in 2011 and publicly traded on NASDAQ as CRWD, the company sells primarily by subscription and also provides incident response services; its systems ingest and analyze nearly 3 trillion security events per day across customer environments.