Kafka Platform Engineer
American Electric Power4 days ago
Columbus, OH, USASenior
Base Salary
$99k - $129k/yr
Responsibilities
- Deploy and manage Kafka connectors, including CDC, JDBC, IBM MQ, and Redshift connectors, in Confluent Kafka environments.
- Containerize and deploy microservices and connectors on OpenShift using Docker, YAML, and Kustomize.
- Implement CI/CD pipelines with GitHub Actions and Argo CD for automated deployments.
- Perform Kafka administration tasks such as topic creation, partitioning, replication, and monitoring.
- Monitor Kafka clusters and connector health using Confluent Control Center, Prometheus, Grafana, Dynatrace, Splunk, and OpenShift-native monitoring.
- Build and maintain custom connectors and producer/consumer applications for specialized data sources and sinks.
- Support production deployments and troubleshoot complex issues across Kafka pipelines and OpenShift environments.
- Deploy Confluent connectors using AWS and Azure cloud services to support real-time data streaming between on-premises systems and cloud or SaaS platforms.
- Collaborate with architects, business teams, product owners, and platform teams to deliver scalable integration solutions.
- Coordinate with co-sourced and offshore teams on development and deployment tasks.
- Conduct code reviews and quality checks while ensuring adherence to standards.
- Automate deployment, monitoring, and operational tasks and own integration deliverables through production deployment.
- Act as a subject matter expert for Kafka-based integrations and cluster administration.
- Apply Kafka security controls, distributed-systems principles, fault-tolerance practices, and multi-region high-availability approaches.
Requirements
- Bachelor's degree in computer science, engineering, or a related technical field is required.
- Seven years of relevant work experience is required; an equivalent combination of education and related experience may be considered.
- Strong hands-on Kafka experience in production environments and understanding of distributed systems, messaging semantics, and fault tolerance.
- Experience deploying Kafka on Kubernetes platforms such as OpenShift and Docker-based environments.
- Experience with multi-region, highly available on-premises deployments and building custom microservices in Python or Java.
- Experience with Kafka security, including TLS, ACLs, and RBAC, and the ability to troubleshoot complex cross-team data-flow issues.
- Preferred experience with Schema Registry, Kafka Connect, MirrorMaker, and Oracle GoldenGate or similar CDC tools.
- Preferred familiarity with enterprise integration platforms and patterns, including APIM, Camunda, webMethods, Oracle SOA, or similar technologies.
- Relevant listed certifications include Confluent Certified Administrator for Apache Kafka, Confluent Certified Developer for Apache Kafka, and Confluent Certified Cloud Operator.
Tech Stack
Amazon RedshiftApache KafkaArgo CDAWSAzureDockerGitHub ActionsGrafanaJavaKubernetesOpenShiftPrometheusPythonSplunk
Categories
Data EngineeringDevOps