Senior Data Engineer - Streaming
The Coca-Cola Company1 day ago
Atlanta, GA, USASenior
Base Salary
$152k - $178k/yr
Responsibilities
- Build a reusable enterprise streaming architecture playbook focused on Kappa patterns, event-driven processing, and near-real-time analytics on Microsoft Fabric.
- Architect, document, and templatize replay and backfill mechanisms using offset resets, checkpoint and state management, and sink rebuild or deduplication strategies.
- Create architectural blueprints and guardrails for sub-second real-time and near-real-time processing SLAs.
- Package stateful processing, watermarking, late-arriving data remediation, and complex windowing into reusable templates.
- Codify effectively-once and end-to-end idempotency patterns with deterministic replay.
- Build metadata-driven frameworks that handle upstream schema drift without disrupting downstream streaming queries.
- Define streaming design patterns across Apache Kafka and Event Hubs and design scalable ingestion into Microsoft Fabric.
- Build dashboards tracking end-to-end data lag, processing throughput, backpressure, and poison-pill routing through dead-letter queues.
Requirements
- 8+ years of experience in data platform architecture, including at least 3+ years architecting and operating production-grade streaming and Kappa architectures.
- 4+ years of experience running Spark Streaming and Apache Kafka pipelines.
- Understanding of Kafka topic partitioning, consumer group mechanics, Avro and JSON schema registries, and high-throughput performance tuning.
- Deep familiarity with Spark Structured Streaming, Kafka Streams, or Apache Flink.
- Strong Scala and/or PySpark experience with stateful Structured Streaming.
- Advanced knowledge of Delta Lake internals, including append-only streaming sinks and checkpointing behavior.
- Demonstrated experience creating developer-facing frameworks, custom connectors, or configuration-driven streaming engines that abstract infrastructure.
- Ability to explain complex streaming topologies and real-time concepts to application developers and executive stakeholders.
- Preferred: Confluent Certified Developer or Kafka Administrator certification.
- Preferred: experience modernizing Lambda architectures toward Kappa-first designs.
- Preferred: hands-on experience with Fabric Real-Time Intelligence, Fabric Eventstreams, and KQL Databases, or equivalent deep experience with Azure Event Hubs, Stream Analytics, or Databricks Delta Live Tables.
Benefits
- United States base pay range is $152,000-$178,300 annually, with benefits including medical, financial, and other offerings depending on the position.
- Annual incentive reference value is 15%.
- Location is Atlanta, United States, with 0%-25% travel required and no relocation provided.
- The position does not offer employment-based visa sponsorship and requires current full-time US work authorization.
- Job posting end date is November 6, 2026.
Tech Stack
Categories
Data Engineering
About The Coca-Cola Company
The Coca-Cola Company is a global beverage manufacturer and brand owner that sells soft drinks, waters, juices, sports drinks, coffee, and dairy to consumers via retail and foodservice, largely through a franchised bottling system. Founded in 1892 and headquartered in Atlanta, it is a public company traded on the NYSE (KO) with products sold in over 200 countries and territories.