2 days ago
Pune, IndiaSenior

Responsibilities

  • Design and operate AWS networking and private connectivity for Databricks.
  • Build and maintain reusable Terraform modules and infrastructure-as-code standards.
  • Administer Databricks capabilities including Unity Catalog, network policies, NCC, and account and workspace settings.
  • Troubleshoot and resolve complex Terraform, networking, and Databricks platform issues.
  • Automate platform operations using Python, SQL, and shell scripting.
  • Drive security, reliability, governance, and operational best practices across the Databricks platform.
  • Partner with engineering, cloud, network, and security teams to deliver a scalable production-ready data platform.

Requirements

  • At least 4 years of AWS experience, including VPC, PrivateLink and endpoint services, NLBs, security groups, IAM policies, and KMS.
  • Strong Terraform experience with modules, remote state, TFE or HCP workspaces, and debugging failed applies.
  • Strong networking knowledge including DNS, TLS, forward proxies, and troubleshooting.
  • Proficiency with Python, SQL, and shell scripting.
  • Experience building and maintaining enterprise-grade applications.
  • Bonus: Databricks administration experience with Unity Catalog, serverless compute, account and workspace settings, network policies, NCC, and private connectivity.
  • Bonus: Experience with MLflow, Feature Store, Model Registry, AI Guardrails, and API integrations.

Tech Stack

AWSDatabricksMLflowPythonSQLTerraform

Categories

Data EngineeringDevOps
Contact me