Senior Spark QA Engineer
InfraCloud Technologies5 months ago
Remote, India or Pune, IndiaSenior
Responsibilities
- Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.
- Execute functional, scalability, and performance testing for Spark workloads.
- Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.
- Conduct benchmarking, validation, and performance analysis of Spark applications.
- Identify bottlenecks and troubleshoot distributed-system issues.
- Lead QA initiatives and mentor team members.
Requirements
- At least 5 years of QA/testing experience with strong hands-on expertise in Apache Spark.
- Experience with functional and performance testing of distributed systems.
- Strong understanding of Spark architecture and optimization techniques.
- Experience with Kubernetes and cloud-based Spark environments.
- Proficiency in Python or Java.
- Experience with automation frameworks and CI/CD pipelines.
Benefits
- Remote work arrangement.
Tech Stack
Categories
About InfraCloud Technologies
InfraCloud Technologies provides cloud-native consulting and engineering services to enterprises adopting Kubernetes, AI/ML platforms, and modern DevOps practices. Its teams design and implement Kubernetes platforms, CI/CD pipelines, infrastructure as code, and observability, and contribute open source tooling around containers and automation; it also offers training and managed services. Headquartered in Pune, India, InfraCloud is a privately held company and part of Improving Enterprises.