1 day ago
Base Salary
$125k - $207k/yr
Responsibilities
- Provide technical leadership and day-to-day direction to a blended onshore/offshore engineering team.
- Design and develop Databricks-based batch and event-driven data solutions, including streaming ingestion, event processing, and downstream event publishing.
- Define and maintain event contracts, schemas, versioning, schema evolution, streaming reliability controls, replay, backfill, and dead-letter handling.
- Develop, maintain, troubleshoot, and optimize data pipelines and models for performance, cost, reliability, and continuity.
- Implement data quality checks, automated testing, monitoring, error handling, incident response, root cause analysis, and service-level operations.
- Apply data governance, privacy, security, PII handling, access controls, and lineage practices through Unity Catalog.
- Build and maintain CI/CD pipelines, infrastructure as code, environment promotion practices, and REST APIs hosted in Kubernetes.
- Partner with Data Science on data and feature pipelines supporting model development and deployment.
- Lead design reviews, establish engineering standards, document systems and procedures, and mentor junior engineers.
Requirements
- Bachelor’s degree in Computer Science, Management Information Systems, Mathematics, Business Analytics, or another technically strong program, or equivalent experience.
- At least 5 years of professional data engineering experience.
- At least 5 years of experience developing with Spark, Databricks, or PySpark, and at least 5 years developing with Python and SQL.
- At least 2 years of experience technically leading or mentoring engineers, including blended onshore/offshore or vendor teams.
- Experience building event-driven or streaming data solutions with Spark Structured Streaming and a messaging platform such as Kafka or Azure Service Bus.
- Understanding of event-driven architecture, publish/subscribe, event sourcing, change data capture, delivery guarantees, idempotent processing, windowing, watermarking, and late or out-of-order events.
- Experience with event schema management using technologies such as Avro, Protobuf, or JSON Schema with a schema registry.
- Experience with Azure data services including ADLS, Azure Key Vault, Azure Kubernetes Service, and Azure Data Factory.
- Experience with Git-based workflows, code review, CI/CD, and infrastructure as code.
- Strong data modeling skills, including dimensional modeling, medallion architecture, Delta Lake, and schema evolution.
- Experience with Databricks Unity Catalog or equivalent data governance and access-control technology.
- At least 1 year of experience building REST APIs and at least 1 year of experience working with Kubernetes.
- Experience with production monitoring, on-call or incident response, and root cause analysis.
- Strong communication skills, business-to-technical translation, influencing without direct authority, and knowledge of Agile principles.
- Experience using AI-assisted development tools with disciplined review of generated code.
- Preferred qualifications include Databricks or Azure certification, legacy ETL or orchestration migration, Databricks Auto Loader, Delta Live Tables or Lakeflow Declarative Pipelines, event-driven API patterns, machine learning workloads or MLOps, and retail, advertising technology, or retail media experience.
Benefits
- Five-day in-office work schedule.
- Medical, dental, vision, wellness, and mental health support.
- 401(k) with Roth option and matching contribution, health savings account with matching contribution, AD&D, and supplemental insurance options.
- Paid time off including 5 weeks of vacation, 7 health and wellness days, 3 floating holidays, and 6 company-paid holidays annually.
- Paid maternity, paternity, and family care leave.
- Eligible for variable compensation in addition to base salary.
Tech Stack
Categories
Data Engineering
About 84.51˚
84.51° is a retail data science, insights, and media company that builds analytics platforms and retail media solutions for The Kroger Co. and consumer packaged goods brands. It commercializes Kroger first-party loyalty data through products such as 84.51° Insights and Kroger Precision Marketing to support targeting, measurement, and merchandising decisions. Founded in 2015 and headquartered in Cincinnati, it operates as a privately held company offering subscriptions, consulting, and managed media services.
