2 days ago
Belgrade, SerbiaSenior
Responsibilities
- Administer, tune, and evolve the Databricks Lakehouse platform, including workspaces, compute policies, Unity Catalog, Lakeflow, and Databricks Asset Bundles.
- Design and manage AWS and Databricks infrastructure as code using Terraform and automated deployment pipelines.
- Own platform security and governance, including IAM, least-privilege access, SSO/SCIM, secrets management, network isolation, private connectivity, encryption, audit logging, lineage, and metadata.
- Build observability, monitoring, alerting, and dashboards for pipeline health, job SLAs, cluster utilization, query performance, and platform costs.
- Design GitLab CI/CD pipelines, deployment patterns, and reusable templates for data engineers shipping pipelines and dbt models.
- Drive capacity planning, incident response, root-cause analysis, reliability improvements, and compute and workload cost optimization.
- Evaluate Databricks and AWS capabilities and propose scalable, secure, automated, and self-service platform improvements.
- Partner with data engineers and data scientists using Python, SQL, dbt, Databricks Workflows, Managed Airflow, Lakeflow, and Fivetran.
Requirements
- 5+ years of experience in platform, DevOps, cloud, or data infrastructure engineering, including significant hands-on experience operating production data platforms.
- Deep hands-on expertise with Databricks workspace and account administration, Unity Catalog, compute and cluster policies, Databricks Asset Bundles, Lakeflow, Databricks Workflows, and system tables.
- Strong experience with AWS infrastructure and services including IAM, S3, VPC/networking, ECS, and Lambda, or equivalent services from other cloud providers.
- Proficiency with Terraform or CloudFormation for infrastructure as code, preferably including the Databricks Terraform provider, module design, state management, and automated plan/apply workflows.
- Strong Python programming skills and expert-level SQL skills for automation, tooling, and platform diagnostics.
- Practical knowledge of least-privilege IAM, secrets management, SSO/SCIM, network isolation, encryption, and audit or compliance requirements.
- Experience implementing observability and monitoring for data-platform metrics, logging, alerting, SLAs, and cost monitoring.
- Working knowledge of dbt and modern ELT patterns, plus experience supporting Managed Airflow or Databricks Workflows.
- Experience building CI/CD pipelines for infrastructure and data workloads, with GitLab CI/CD preferred.
- A track record of proposing and implementing improvements that make systems more scalable, secure, and self-service.
- Experience with other data warehouse, data lake, or data lakehouse platforms is an asset.
- Fivetran or similar managed ingestion tools, data quality and monitoring frameworks, Databricks or AWS certifications, and FinOps practices are nice-to-have qualifications.
About Rivian
Rivian designs and manufactures electric vehicles, including the R1T pickup, R1S SUV, and commercial delivery vans, selling directly to consumers and fleet operators. Founded in 2009 and headquartered in Irvine, California, it operates a manufacturing plant in Normal, Illinois, and is publicly traded on NASDAQ. The company also develops in-vehicle software and electrical architecture and offers charging and service support for its owners.