Senior Data Infrastructure Engineer

2 days ago

Base Salary

$150k - $200k/yr

Responsibilities

Build and operate the Apache Iceberg lakehouse on S3, including table design, partitioning, compaction, and migration from Redshift.
Own end-to-end ingestion through Flink CDC, Kafka/MSK, Iceberg, RudderStack, Fivetran, and DMS while meeting freshness and reliability SLAs.
Run and evolve compute and orchestration with Apache Kyuubi, dbt-Spark, EKS, Airflow, autoscaling, spot strategy, and cost optimization.
Develop self-service tooling, libraries, and templates for analytics engineers and data scientists.
Build staging environments, CI testing against staging, schema-change detection, gated promotion, and canary deployments.
Own platform governance and access controls, including Lake Formation, row and column RBAC, StrongDM, SSO, audit logging, and PII handling.
Own observability, lineage, alerting, incident response, root-cause analysis, and durable reliability improvements.
Drive infrastructure-as-code and automation using Terraform, GitLab CI, and GitOps.
Partner with backend engineering and analytics consumers to shape data contracts and platform expectations.

At least 4 years of experience in data engineering, data platforms, or infrastructure engineering; 6 or more years for the senior level.
Strong Python and SQL skills with experience building frameworks and tooling used by other engineers.
Production experience operating orchestration frameworks such as Airflow, Dagster, or Prefect at meaningful scale.
Hands-on Apache Spark experience and strong distributed-systems fundamentals.
Deep AWS experience, including S3, EKS/ECS, IAM, and Glue/Athena or equivalent services.
Experience with CI/CD, infrastructure as code, Terraform, GitOps, Kubernetes, and Docker.
Track record owning SLAs, monitoring, alerting, on-call operations, and post-incident hardening.
Daily hands-on use of AI coding tools such as Claude Code or Cursor.
Strong cross-functional communication skills.
Preferred experience with Apache Iceberg, Delta Lake, Hudi, lakehouse migrations, Kafka/MSK, Flink, CDC, Kinesis, Lake Formation, Unity Catalog, RBAC and masking, dbt-Spark, Monte Carlo, AI/LLM workloads, low-latency analytics engines, and open-source data infrastructure contributions.

501-1,000 employees