Senior Testing Engineer
Robots and Pencils1 day ago
Remote, ColombiaMid Level / Senior
Responsibilities
- Design and develop scalable automation frameworks for UI, API, integration, and AI testing.
- Integrate automated tests into CI/CD pipelines and drive quality gates.
- Develop AI-specific test strategies and validate ML models using accuracy, precision, recall, bias, and drift metrics.
- Test data pipelines and feature engineering processes, and validate LLM responses for hallucination risk and consistency.
- Plan tests, review automation code, and uphold testing best practices.
- Monitor model quality in production and support release validation and quality assurance.
- Collaborate with developers, ML engineers, DevOps, and product teams across the full SDLC.
- Lead quality efforts, contribute to QA standards, improve coverage and reliability, and begin mentoring junior engineers.
Requirements
- 3–4+ years of experience in QA, data QA, or AI testing.
- Strong programming skills in Python and at least one additional language such as Java or JavaScript.
- Hands-on experience designing and maintaining automation frameworks.
- Strong understanding of the ML lifecycle and model evaluation metrics.
- Knowledge of LLM validation, hallucination testing, and data pipeline validation.
- Experience with CI/CD pipelines, API automation, and version control.
- Familiarity with MLOps pipelines and production model monitoring.
- Experience with Agile methodologies and test management tools such as Jira and TestRail.
- Demonstrable use of AI-forward tools such as Claude and Cursor.
- Experience with mobile automation, performance testing, LLM testing frameworks, prompt engineering, or cloud ML platforms is a plus.
Tech Stack
Apache JMeterAppiumCucumberCypressGitGitHub ActionsGitLab CI/CDJavaJavaScriptJenkinsPostmanpytestPythonSeleniumTestNG
Categories