18 hours ago
Seattle, WA, USA +3 moreMid Level
H1B sponsor
Base Salary
$131k - $192k/yr
Responsibilities
- Build and operate an in-house Apache Spark platform spanning runtime, scheduler, reliability, and user-facing tooling.
- Develop multi-tenant scheduling, executor bin-packing, and cost-aware workload placement.
- Own cluster lifecycle automation for provisioning, upgrades, capacity changes, and node-failure handling.
- Build observability and incident automation to improve platform debuggability and on-call sustainability.
- Partner with senior engineers on shuffle, runtime, and distributed-systems architecture work.
- Support and operate thousands of Spark cluster deployments for company-wide consumers.
Requirements
- Bachelor’s, master’s, or PhD in Computer Science or equivalent.
- At least 2 years of industry experience operating production distributed systems.
- Experience operating Apache Spark at scale on Amazon EMR, Databricks, or an in-house deployment, particularly platform operations rather than authoring individual Spark jobs.
- Hands-on experience operating production systems on Kubernetes, including controllers, operators, custom resources, and multi-tenant cluster failure modes.
- Familiarity with batch or big-data schedulers such as YuniKorn, Volcano, or Kueue, and/or the Spark-on-Kubernetes operator.
- Familiarity with Prometheus, OpenTelemetry, distributed tracing, structured logging, and defining SLOs and SLIs.
- Comfort working with cloud infrastructure, preferably AWS, including VPC networking, instance lifecycle, spot or preemptible markets, and autoscaling primitives.
- Professional experience with Python, Go, Scala, or Java, along with SQL fluency.
- Willingness to work incrementally, measure outcomes, and reduce operational toil.
- Located in or willing to relocate to the Bay Area, Seattle, or New York City.
Benefits
- Hybrid position requiring location in San Francisco, Sunnyvale, Seattle, or New York City.
- Regular employees receive medical, dental, vision, disability, and basic life insurance benefits.
- Benefits include a 401(k) plan with employer matching, paid parental leave, paid time off, paid sick leave, paid holidays, wellness benefits, commuter benefits match, family-forming assistance, and a mental health program.
- Salaried roles receive flexible paid time off or vacation plus 80 hours of paid sick time per year.
- The role includes opportunities for equity grants.
Tech Stack
Categories
About DoorDash
DoorDash builds a marketplace and logistics platform that connects consumers with local restaurants and retailers for on-demand delivery and pickup, powered by a network of independent Dashers. It earns through delivery fees, merchant commissions, advertising, DashPass subscriptions, and white-label fulfillment via DoorDash Drive. Founded in 2013 and headquartered in San Francisco, the public company operates across the U.S. and selected international markets, and also researches automation through DoorDash Labs.