Kafka Engineer
Fractal Analyticsover 1 year ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, deploy, scale, and manage Kafka and Confluent infrastructure for high-throughput streaming applications.
- Implement and maintain Kafka Connect, Schema Registry, KSQL/KSQLDB, FlinkSQL, and Kafka Streams components.
- Ensure Kafka cluster availability, performance, tuning, monitoring, maintenance, and troubleshooting.
- Integrate streaming pipelines with data lakes and warehouses in collaboration with data engineers and backend teams.
- Configure and enforce Kafka security mechanisms including SASL, SSL, ACLs, and RBAC.
- Create and manage CI/CD pipelines for Kafka components using Git and Jenkins.
- Monitor Kafka systems using Confluent Control Center and DataDog.
Requirements
- Hands-on experience with Apache Kafka and Confluent Platform.
- Experience with Kafka Connect, Kafka Streams, Schema Registry, KSQL/KSQLDB, and FlinkSQL.
- Strong background in Java or Python and a good understanding of stream processing and event-driven architecture.
- Hands-on experience with AWS, Azure, or GCP, especially Confluent Cloud.
- Proficiency in Kafka cluster deployment, tuning, and maintenance.
- Familiarity with Git, Helm, Kubernetes, and CI/CD.
- Knowledge of Terraform, Pulumi, or other infrastructure-as-code tools is preferred.
- Exposure to DataOps and data engineering ecosystems is preferred.
Tech Stack
Categories
Data EngineeringDevOps