about 3 hours ago
Base Salary
$131k - $192k/yr
Responsibilities
- Set the multi-year technical direction for the Spark-on-Kubernetes platform.
- Own complex distributed-systems problems related to shuffle architecture and multi-tenant scheduling.
- Collaborate with the Engineering Manager on technical roadmap and team development.
- Mentor team members through design reviews and raise the quality of deliverables.
- Represent Spark Platform in cross-team architecture forums.
Requirements
- B.S., M.S., or PhD in Computer Science or equivalent.
- 6+ years of experience designing and operating distributed systems at scale.
- Deep experience with Apache Spark internals and operations at platform scale.
- Production experience with remote shuffle systems and big-data schedulers.
- Strong fluency in operating workloads on Kubernetes in production.
- Familiarity with data lake table formats like Apache Iceberg or Delta Lake.
- Proven track record as a technical leader on a platform team.
- Professional experience with Scala, Java, Python, or Go; strong SQL skills.
- Willingness to relocate to the Bay Area, Seattle, or NYC.
Benefits
- Comprehensive benefits package including medical, dental, and vision.
- 401(k) plan with employer matching.
- 16 weeks of paid parental leave.
- Flexible paid time off and paid sick leave.
- Wellness benefits and commuter benefits match.
- Mental health program and family-forming assistance.