Data Platform Engineer
Hadrian Automation2 months ago
Base Salary
$170k - $300k/yr
Responsibilities
- Build ingestion, CDC, and streaming capabilities for transactional data, events, telemetry, and files while managing ordering, deletes, retries, replay, idempotence, and backpressure.
- Define versioned data and event contracts with upstream teams, including testing and service targets for freshness, completeness, and correctness.
- Model telemetry, quality events, work orders, and operational data with explicit grain, identity, time, provenance, and history.
- Own Dagster orchestration, dbt transformation, backfills, lineage, observability, data CI/CD, and offline feature datasets for ML.
- Acquire and integrate data from machines, PLCs, historians, OPC-UA, MTConnect, MQTT, and other factory sources in collaboration with Manufacturing Operations and Infrastructure.
- Enable reliable datasets for Analytics, Data Science, Operations Research, machine-learning systems, scheduling, and reporting.
Requirements
- Experience building and operating production data infrastructure or distributed data systems, including on-call ownership and recovery efforts.
- Strong production Python, advanced SQL, and data-modeling skills, including incremental processing, temporal data, and schema evolution.
- Experience with Kafka or another event-streaming platform, plus CDC or other stateful incremental pipelines.
- Experience operating Snowflake and a lakehouse table format such as Iceberg, Delta, or Hudi, including partitioning and compaction.
- Experience with orchestration tools such as Dagster, Airflow, Argo, or Prefect; dbt or similar transformation frameworks; and Kubernetes or infrastructure as code.
- Strong judgment regarding data contracts, failure modes, and downstream analytics, ML, and operational-system requirements.
- Preferred experience running Snowflake and Iceberg together or designing hybrid warehouse and lakehouse architectures.
- Preferred production experience with PeerDB, Debezium, Flink, Spark Structured Streaming, Redpanda, Bufstream, or similar CDC and streaming systems.
- Preferred proficiency with ClickHouse or another low-latency analytical database, including performance tuning and lifecycle management.
- Industrial or edge data collection experience with OPC-UA, MTConnect, MQTT, historians, or PLCs is advantageous.
- Background in performance-sensitive data systems built with Go, Rust, or Scala; regulated-environment experience; or contributions to dbt, Dagster, Iceberg, or related projects is advantageous.
- Applicants must meet applicable ITAR eligibility requirements or be eligible to obtain the required U.S. Department of State authorizations.
Benefits
- Medical, dental, vision, and life insurance plans for full-time employees.
- 401k plan.
- Relocation support may be provided in certain situations based on business need.
- Flexible vacation policy.
- Equity.
Tech Stack
Categories
BackendData Engineering
About Hadrian Automation
Hadrian Automation builds autonomous, software-driven factories that manufacture precision components for aerospace and defense programs, including rockets, satellites, aircraft, and ships. The company operates a Factory-as-a-Service platform, combining AI, robotics, and full-stack machining and fabrication to deliver critical parts and production capacity. Founded in 2021 and headquartered in Los Angeles, it is privately held and expanding capabilities across welding, casting, forging, electronics, and additive manufacturing.