Sr Software Engineer, Infrastructure
Databricks6 months ago
Base Salary
$136k - $187k/yr
Responsibilities
- Design and deploy production-grade AWS infrastructure using Terraform or Pulumi.
- Manage and scale containerized workloads on AKS or EKS, with emphasis on cluster security and resource efficiency.
- Architect CI/CD pipelines with GitHub Actions and manage GitHub-hosted and self-hosted runners.
- Build infrastructure that enables secure logging and metrics by default for internal applications.
- Develop internal CLI tools, AI plugins, and automation scripts to improve developer workflows and operational efficiency.
- Integrate infrastructure with data pipelines and observability platforms.
- Collaborate with Security, Engineering, Infrastructure, and Support stakeholders.
- Conduct code reviews, document solutions and failure-triage playbooks, and mentor junior engineers.
Requirements
- 5+ years of production-level software engineering experience.
- Strong Python proficiency is required.
- Expert-level proficiency with Terraform or Pulumi, with Pulumi preferred.
- Hands-on experience with AWS or Azure/GCP, Kubernetes, and Docker.
- Experience building and troubleshooting integrations between infrastructure, data pipelines, and observability platforms.
- Advanced knowledge of GitHub Actions and GitHub Runners.
- Hands-on observability experience with logging, metrics, tracing, and tools such as Datadog, Prometheus, or ELK.
- Proficiency with distributed systems concepts such as Kafka or messaging queues.
- Ability to work independently, take ownership of ambiguous projects, and execute with minimal guidance.
Benefits
- Hybrid work arrangement in San Francisco, CA
- Comprehensive employee benefits and perks, with region-specific details available
- Potential eligibility for annual performance bonus and equity
Tech Stack
Apache KafkaApache SparkAWSAzureDatadogDockerGitHub ActionsGoogle Cloud PlatformKubernetesMLflowPrometheusPythonTerraform
Categories
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.