3 months ago
Base Salary
$131k - $192k/yr
Responsibilities
- Set the multi-year technical direction for the in-house Spark-on-Kubernetes platform.
- Own distributed-systems architecture across the runtime, shuffle service, scheduler, performance, and reliability layers.
- Partner with the Engineering Manager on technical roadmaps, hiring, interview design, and team structure.
- Mentor engineers through design reviews and raise the quality bar for shipped systems.
- Represent Spark Platform in cross-team architecture forums and influence how data, analytics, and ML workloads use the platform.
Requirements
- Bachelor’s, master’s, or PhD in Computer Science or equivalent.
- 6+ years of industry experience designing and operating distributed systems at scale.
- Deep hands-on Apache Spark experience covering internals, query execution, shuffle, and the executor/driver model at platform scale.
- Experience operating Spark on Amazon EMR, Databricks, or an in-house deployment, with emphasis on platform operations.
- Production experience with remote or external shuffle systems, batch or big-data schedulers, or observability and SRE patterns for distributed compute platforms.
- Strong production Kubernetes experience, including operator patterns, executor pod lifecycle, network topology, and multi-tenant failure modes.
- Familiarity with Apache Iceberg or Delta Lake and query or SQL engines that read them.
- Track record of technical leadership, mentoring, setting direction, and partnering with management on roadmaps and hiring.
- Professional experience with Scala, Java, Python, or Go, plus strong SQL.
- Located in or willing to relocate to the Bay Area, Seattle, or New York City.
Benefits
- Hybrid position requiring location in San Francisco, Sunnyvale, Seattle, or New York City, or relocation to the Bay Area, Seattle, or NYC.
- Comprehensive medical, dental, and vision benefits.
- 401(k) plan with employer matching.
- 16 weeks of paid parental leave.
- Flexible paid time off and 80 hours of paid sick time annually for salaried employees.
- Wellness, commuter benefits match, and mental health benefits.
- 11 paid holidays, disability insurance, basic life insurance, and family-forming assistance.
- Equity grant opportunities in addition to base salary.
Tech Stack
About DoorDash
DoorDash builds a marketplace and logistics platform that connects consumers with local restaurants and retailers for on-demand delivery and pickup, powered by a network of independent Dashers. It earns through delivery fees, merchant commissions, advertising, DashPass subscriptions, and white-label fulfillment via DoorDash Drive. Founded in 2013 and headquartered in San Francisco, the public company operates across the U.S. and selected international markets, and also researches automation through DoorDash Labs.