Senior Big Data Engineer
Qualys, Inc.3 months ago
Pune, IndiaSenior
Responsibilities
- Design, build, and maintain high-throughput data platform and pipeline components using Spark and Kafka.
- Debug, tune, and optimize Spark jobs and Kafka pipelines for growing data volumes.
- Scale data platform infrastructure to process high-volume and high-throughput transaction workloads.
- Collaborate with product, professional services, and sales engineering teams on architecture design and technical trade-offs.
- Establish technical standards, system documentation, and engineering best practices.
- Mentor junior and mid-level engineers on complex technical issues and pipeline performance.
Requirements
- At least 5 years of experience in data engineering or big data platform development.
- Bachelor's degree in Computer Science, Computer Engineering, or a related technical field.
- Demonstrated experience designing, building, and optimizing distributed pipelines using Apache Spark and Apache Kafka.
- Hands-on experience scaling data platform infrastructure for high-volume and high-throughput transaction workloads.
- Experience with high-volume, multi-tenant enterprise platform environments is preferred.
- Proficiency in Scala, Java, or Python for data processing applications is preferred.
- Proven experience mentoring junior or mid-level engineering team members is preferred.
Tech Stack
Categories
Data Engineering
About Qualys, Inc.
Qualys builds a cloud-based security and compliance platform used by enterprises to automate vulnerability management, IT asset visibility, web application scanning, and policy/PCI compliance across on‑prem and cloud environments. Delivered as subscription SaaS, its Enterprise TruRisk Platform uses a single agent to provide continuous security telemetry for endpoints, servers, containers, and clouds. Founded in 1999 and headquartered in Foster City, California, Qualys is a public company trading on NASDAQ (QLYS).