about 3 hours ago
Seattle, WA, USA +3 moreMid Level / Senior
H1B Sponsor
Base Salary
$131k - $192k/yr
Responsibilities
- Build and operate an in-house Spark platform at company-wide scale.
- Drive multi-tenant scheduling and executor bin-packing.
- Own cluster lifecycle automation including provisioning and upgrades.
- Develop observability and incident automation for platform sustainability.
- Collaborate with senior engineers on runtime and architecture work.
Requirements
- B.S., M.S., or PhD in Computer Science or equivalent.
- 24+ years of industry experience with production distributed systems.
- Experience operating Apache Spark at scale with a focus on platform operations.
- Hands-on experience with Kubernetes and its failure modes in multi-tenant clusters.
- Familiarity with batch or big-data schedulers and Spark-on-Kubernetes operator.
- Knowledge of observability stacks and defining SLOs and SLIs.
- Comfortable working in a cloud environment, preferably AWS.
- Professional experience with Python, Go, Scala, or Java; SQL fluency.
- A bias toward incremental rollout and reducing toil.
- Willingness to relocate to the Bay Area, Seattle, or NYC.
Benefits
- Comprehensive benefits package including 401(k) with employer matching.
- 16 weeks of paid parental leave.
- Wellness benefits and commuter benefits match.
- Paid time off and paid sick leave.
- Medical, dental, and vision benefits.
- 11 paid holidays and disability insurance.