Lead Software Engineer - Performance
Qualys, Inc.2 hours ago
Pune, IndiaStaff+
Responsibilities
- Own performance strategy across Hadoop, Spark, Kafka, Elasticsearch/OpenSearch, big-data components, and APIs for each release.
- Define, develop, and execute performance, load, stress, soak, and benchmark test plans.
- Create production-like performance scenarios for data pipelines and microservices and build a synthetic-data generation framework.
- Identify bottlenecks, resource contention, latency issues, and scaling opportunities using performance and observability tools.
- Analyze and tune Spark jobs, Kafka topics and partitions, Elasticsearch queries, and API endpoints.
- Collaborate with developers, architects, and infrastructure teams to incorporate performance feedback into system design and implementation.
- Build a performance-testing framework integrated with CI/CD pipelines.
- Establish and track throughput, latency, CPU, memory, garbage-collection, and other service-level objectives.
- Create performance dashboards and visualizations using Prometheus/Grafana, Kibana, or equivalent tools.
- Document performance findings and recommendations for engineering teams and leadership.
- Recommend performance and cost optimizations to development and platform groups.
- Contribute to feature development and software fixes in addition to performance benchmarking.
Requirements
- Bachelor’s degree in computer science, engineering, or a related field.
- 8+ years of overall experience in distributed systems and backend performance engineering.
- 4+ years of Java development experience with microservices architecture.
- Proficiency in Python and Bash scripting for automation and test-data generation.
- 4+ years of hands-on Apache Spark experience, including performance tuning, memory management, and DAG optimization.
- 3+ years of Kafka experience, including topic optimization, producer and consumer tuning, and lag monitoring.
- 3+ years of Elasticsearch or OpenSearch experience, including query profiling, indexing strategies, and cluster optimization.
- 3+ years of experience with JMeter or similar performance-testing tools.
- Hands-on experience with Spring and Hibernate and strong programming and system-design skills.
- Deep understanding of middleware and microservices performance, including REST APIs.
- Strong knowledge of profiling, debugging, and observability tools such as Spark UI, Athena, Grafana, and ELK.
- Experience designing and running large-scale benchmarks for high-throughput environments measured in petabytes.
- Experience with containerized workloads and performance testing in Kubernetes and Docker environments.
- Understanding of cloud-native architecture, OCI, Linux performance improvements, data lake architecture, caching solutions, and message queues.
- Familiarity with CI/CD integration for performance testing, including Jenkins or GitHub.
- Strong communication skills and experience influencing cross-functional engineering teams.
- Prior experience with an analytics platform on big data is a plus.
Tech Stack
Apache HadoopApache JMeterApache KafkaApache SparkBashDockerElasticsearchGrafanaJavaJenkinsKibanaKubernetesLinuxPrometheusPython