AI engineer-Data Databricks Pltform
Ecolab Inc.1 day ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, configure, and maintain Databricks workspaces across development, testing, and production environments.
- Manage Unity Catalog, clusters, SQL Warehouses, compute policies, workspace configurations, and access controls.
- Create reusable platform standards, frameworks, templates, deployment automation, and engineering accelerators using Terraform and CI/CD pipelines.
- Build scalable batch, streaming, API, database, and file-based data ingestion pipelines using Databricks Lakehouse architecture, Delta Lake, Structured Streaming, and medallion patterns.
- Integrate Databricks with Snowflake, SAP, Oracle, SQL Server, Azure Data Factory, Kafka, cloud storage, LLMs, vector databases, AI gateways, and enterprise AI platforms.
- Enable ML and generative AI workloads through MLflow, model lifecycle management, feature stores, model serving, and scalable development environments.
- Establish CI/CD, automated testing, deployment, monitoring, rollback, release management, and environment promotion processes for data and AI workloads.
- Implement security, RBAC, data masking, encryption, audit logging, Unity Catalog governance, platform hardening, and vulnerability remediation.
- Implement observability, logging, alerting, performance monitoring, incident management, and production support processes.
- Optimize platform performance, cost, scalability, reliability, security, and governance while mentoring junior engineers and contributing to architecture and roadmap planning.
Requirements
- Bachelor's or master's degree in computer science, engineering, IT, or a related field.
- At least 5 years of experience in data, platform, or cloud engineering.
- At least 3 years of hands-on experience with the Databricks Lakehouse Platform.
- Strong expertise in Delta Lake, Unity Catalog, MLflow, Databricks Workflows, Structured Streaming, PySpark, and Spark SQL.
- Experience with Azure, AWS, or GCP cloud platforms.
- Proficiency in Python and SQL.
- Strong knowledge of DevOps, CI/CD, Infrastructure as Code, and platform automation.
- Hands-on experience with Terraform, GitHub Actions, Azure DevOps, or equivalent CI/CD tools.
- Preferred experience with generative AI, agentic AI, LLM-based applications, Snowflake, enterprise AI platforms, Kubernetes, Docker, APIs, Kafka, and event-driven architectures.
- Databricks Certified Professional or Associate certification and experience supporting enterprise-scale Data & AI platforms are preferred.
Tech Stack
Apache KafkaAWSAzureDatabricksDockerGitHub ActionsGoogle Cloud PlatformKubernetesMicrosoft SQL ServerMLflowPythonSnowflakeSQLTerraform
Categories
Data EngineeringDevOps
About Ecolab Inc.
Ecolab Inc. builds and sells water treatment chemicals and equipment, hygiene and infection-prevention products, and on-site service and digital monitoring for industrial and institutional customers. A NYSE-listed public company founded in 1923 and headquartered in St. Paul, Minnesota, it serves food, healthcare, life sciences, hospitality, manufacturing, and energy markets worldwide; its portfolio includes Nalco Water and, in data centers, CoolIT liquid-cooling solutions.