Software Engineer in Test II
AlertMedia4 hours ago
Mexico City, MexicoMid Level
Responsibilities
- Build and maintain evaluation suites for AI features, including regression baselines, scoring, and end-to-end agent behavior testing.
- Test AI guardrails, human approval gates, adversarial inputs, prompt injection, and safe failure behavior.
- Design, execute, and maintain automated test coverage for web and mobile applications.
- Perform manual, exploratory, functional, regression, API, integration, and performance testing.
- Identify, document, prioritize, and track software defects through resolution.
- Partner with Engineering, Product, and Customer Success to troubleshoot issues and improve product quality.
- Contribute to QA strategy, processes, tools, and organization-wide testing practices.
- Mentor team members and improve testing standards, automation practices, and AI-powered QA workflows.
Requirements
- 5+ years of experience in QA, SDET, or software testing roles.
- Experience with LLM evaluation techniques such as curated case sets, rubric or model-graded scoring, and regression baselines.
- Hands-on experience testing agentic systems, tool or function calling, multi-step workflows, MCP or similar integrations, and related failure modes.
- Strong understanding of API, integration, and distributed systems testing.
- Experience designing scalable automation frameworks and test strategies using tools such as Selenium, Cypress, Playwright, or Gherkin.
- UI automation testing experience with Python or JavaScript.
- Familiarity with CI/CD workflows and tools such as Git and Jenkins.
- Familiarity with SQL and/or NoSQL databases.
- Experience using AI-powered engineering tools such as Codex, Claude, or Cursor.
- Strong communication, collaboration, English proficiency, and cross-functional working skills.
- Experience mentoring teammates or serving as a QA thought partner is preferred.
- A bachelor’s degree is preferred.
Benefits
- Competitive base salary paid through bi-monthly payroll, annual company bonus, and employer-paid medical, dental, and vision coverage.
- Parental leave, Christmas bonus, vacation and holiday policy, and additional vacation bonus.
- Monthly work-from-home stipend and necessary equipment including a laptop, monitor, and accessories.
- Continuous learning, mentorship, tech talks, and individualized career development support.
- Fully remote work within Mexico or hybrid work one to two days per week at the Reforma-area Mexico City office.
Tech Stack
Categories
About AlertMedia
AlertMedia builds a SaaS risk intelligence and emergency communication platform used by organizations to monitor threats, notify employees, and coordinate incident response. Its products include mass notification, threat monitoring and intelligence, and employee safety features such as lone-worker monitoring. Founded in 2013 and headquartered in Austin, Texas, the company is privately held under Vista Equity Partners and serves enterprises across industries and geographies.