2 days ago
San Mateo, CA, USASenior
Base Salary
$243k - $295k/yr
Responsibilities
- Design, build, and operate Spark job submission services, APIs, scheduling integrations, and internal Spark libraries.
- Launch and manage interactive Spark notebook compute using Spark Connect and EKS with appropriate isolation, resource limits, and cost controls.
- Automate EKS cluster fleet management, including blue-green rollouts, AMI validation, Kubernetes upgrades, and node lifecycle operations.
- Develop Spark auto-tuning tools that use job history and event logs to improve performance and reduce compute costs.
- Operate and upgrade YuniKorn-based Kubernetes batch scheduling, including queues, quotas, tenant fairness, and version upgrades.
- Evaluate and benchmark emerging Spark execution technologies such as Apache Comet and make data-backed adoption recommendations.
- Define SLOs and observability, improve reliability, reduce incidents and operational toil, and control platform spending.
- Partner with data engineering, ML, analytics, and infrastructure teams and mentor engineers across cross-cutting technical initiatives.
Requirements
- 5+ years of experience building and operating production-grade distributed systems or infrastructure platforms.
- Hands-on experience running Apache Spark on Kubernetes, including EKS, GKE, or self-managed Kubernetes, executor and driver behavior, resource tuning, and troubleshooting.
- Strong experience operating production Kubernetes clusters at scale, including upgrades, node lifecycle, autoscaling, and operators or controllers.
- Strong proficiency in at least one of Go, Java, Scala, or Clojure, with experience writing clean, tested, maintainable backend code.
- Experience with AWS or GCP equivalents, infrastructure-as-code, production operations, on-call, incident response, and observability.
- Ability to influence technical decisions, mentor engineers, and drive alignment across teams.
- B.S. in Computer Science or equivalent practical experience.
- Preferred experience includes YuniKorn, Volcano, Kueue, Spark Connect, Livy, Kyuubi, Spark Operator, Karpenter, Helm, Terraform, Argo, Spark event-log tooling, internal Spark builds, native execution engines, JVM performance tuning, YARN/Hadoop migrations, open-source data or Kubernetes contributions, or consumer-internet scale systems.
Benefits
- Full-time employees are eligible for equity compensation and company benefits.
- San Mateo headquarters roles are onsite Tuesday through Thursday, with optional presence Monday and Friday.
- The posting includes an annual base salary range of $243,290–$295,250 for roles based at the San Mateo headquarters.
Tech Stack
Apache HadoopApache SparkArgo CDAWSClojureGoGoogle Cloud PlatformHelmJavaKubernetesPythonScalaTerraformYarn
Categories
Data EngineeringDevOps
About Roblox
Roblox builds a platform for user‑generated 3D experiences, offering Roblox Studio for creators and a social marketplace where players explore and interact across devices. Its business model centers on virtual currency (Robux) and subscriptions, platform fees paid to creators, and an expanding advertising offering. Founded in 2004 and headquartered in San Mateo, California, Roblox Corporation is publicly traded on the NYSE under the ticker RBLX.