AI engineer-Databricks Data Platform
Ecolab Inc.3 days ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, configure, and maintain Databricks workspaces across development, testing, and production environments.
- Manage Unity Catalog, clusters, SQL Warehouses, compute policies, workspace configurations, and access controls.
- Build scalable batch, streaming, API, database, and file-based data ingestion pipelines using Databricks Lakehouse patterns.
- Integrate Databricks with Snowflake, SAP, Oracle, SQL Server, Azure Data Factory, Kafka, and cloud storage services.
- Enable machine learning and generative AI workloads through MLflow, feature stores, model lifecycle management, and model serving.
- Establish CI/CD, DataOps, MLOps, automated testing, deployment, rollback, release management, and environment promotion processes.
- Implement infrastructure as code, reusable deployment frameworks, platform automation, security controls, governance, data masking, encryption, and audit logging.
- Configure observability, logging, alerting, performance monitoring, incident management, and production support processes.
- Optimize platform performance, cost, scalability, reliability, security, and compliance.
- Collaborate with architects, data engineers, AI engineers, cybersecurity teams, and business stakeholders; provide technical guidance and mentor junior engineers.
Requirements
- Bachelor’s or master’s degree in computer science, engineering, IT, or a related field.
- At least 5 years of experience in data, platform, or cloud engineering.
- At least 3 years of hands-on experience with the Databricks Lakehouse Platform.
- Strong expertise in Delta Lake, Unity Catalog, MLflow, Databricks Workflows, Structured Streaming, PySpark, and Spark SQL.
- Experience with Azure, AWS, or GCP cloud platforms.
- Proficiency in Python and SQL.
- Strong knowledge of DevOps, CI/CD, infrastructure as code, and platform automation.
- Hands-on experience with Terraform, GitHub Actions, Azure DevOps, or equivalent CI/CD tools.
- Preferred: experience with Generative AI, Agentic AI, LLM-based applications, Snowflake, enterprise AI platforms, Kubernetes, Docker, APIs, Kafka, and event-driven architectures.
- Preferred: Databricks Certified Professional or Associate certification and experience supporting enterprise-scale Data & AI platforms.
Tech Stack
Apache KafkaAWSAzureDatabricksDockerGitHub ActionsGoogle Cloud PlatformKubernetesMicrosoft SQL ServerMLflowPythonSnowflakeSQLTerraform
Categories
Data EngineeringDevOps
About Ecolab Inc.
Ecolab Inc. builds and sells water treatment chemicals and equipment, hygiene and infection-prevention products, and on-site service and digital monitoring for industrial and institutional customers. A NYSE-listed public company founded in 1923 and headquartered in St. Paul, Minnesota, it serves food, healthcare, life sciences, hospitality, manufacturing, and energy markets worldwide; its portfolio includes Nalco Water and, in data centers, CoolIT liquid-cooling solutions.