1 month ago
Base Salary
$166k - $207k/yr
Responsibilities
- Lead the technical direction, roadmap, and production operations of the Airflow and Astronomer platform.
- Build reusable paths and tooling for authoring, testing, deploying, and operating workflows.
- Design and operate streaming and change data capture pipelines for analytics and product use cases.
- Improve pipeline resilience for retries, duplicates, schema changes, late events, backfills, and replay.
- Strengthen cloud infrastructure for data workloads using Kubernetes, Terraform, IAM, secrets, CI/CD, and cost-aware capacity management.
- Lead cross-team decisions on data contracts, interfaces, operational ownership, and scalable platform architecture.
- Mentor engineers, partner with application, analytics, ML, and cloud teams, and participate in on-call support.
- Build data and orchestration foundations for batch inference, evaluation datasets, agent workflows, and production data access.
Requirements
- Strong Python software engineering skills and experience building and operating production data or distributed systems.
- Deep hands-on Airflow experience including scheduling, execution behavior, deployment, scaling, upgrades, monitoring, and debugging.
- Experience with event-driven or streaming data infrastructure, message brokers or managed event buses, and stream processing systems.
- Understanding of change data capture, delivery guarantees, idempotency, ordering, schema evolution, recovery, and replay.
- Experience with cloud infrastructure and infrastructure as code, including containers, Kubernetes, CI/CD, access controls, and production observability.
- Track record of leading ambiguous infrastructure initiatives, setting technical direction, making architecture tradeoffs, and driving adoption.
- Clear communication and experience partnering across teams while owning systems through production support.
- Extra-credit experience with GCP services such as Pub/Sub, Dataflow, Datastream, BigQuery, GKE, and Cloud Storage.
- Extra-credit experience with Astronomer, Apache Beam, Terraform, Spacelift, Datadog, dbt, Spark, Dataproc, batch and low-latency consumers, or ML and AI workloads.
Benefits
- Equity, competitive compensation, 401(k) match, financial coaching, paid parental leave, fertility benefits, parental coaching, medical, dental, vision, mental health support, and a $500 wellness stipend.
- A $2,000 learning stipend and ongoing development opportunities.
- Remote work support including internet and commuting benefits, plus free lunch and gym access in the San Francisco office.
- Flexible PTO, 15 holidays, 2 flex days, team outings, and referral bonuses.
- Benefits listed apply to full-time US employees.
Tech Stack
Categories
Data Engineering
About Handshake
Handshake builds a career network and recruiting platform that connects college students and recent grads with employers, sold as SaaS to universities and subscriptions/solutions to employers. Founded in 2014 and headquartered in San Francisco, it serves 1,600+ educational institutions and over 1 million employers. The company also operates Handshake AI, which partners with frontier AI labs on human data collection and evaluations for model training and post-training workflows.