Databricks - Tech Lead II - GR - 40516-75564 - JR206256
Elevance Health21 hours ago
Bengaluru, India or Hyderābād, IndiaStaff+
Responsibilities
- Lead development and implementation of scalable data engineering solutions.
- Develop end-to-end batch and real-time data pipelines using Databricks, Python, Spark, and PySpark.
- Design scalable data lakehouse architectures and Iceberg-based data lakes.
- Build data quality, governance, metadata, security, and observability frameworks.
- Optimize Snowflake workloads, data models, queries, performance, and cost.
- Design and implement cloud-native data solutions using AWS, Azure, and GCP services.
- Review code, establish engineering best practices, and guide performance and reliability improvements.
- Support platform modernization and cloud migration initiatives.
- Partner with architects, product owners, and business stakeholders to translate requirements into technical solutions.
- Provide effort estimation and delivery planning and help resolve complex production issues.
Requirements
- B.Tech degree or MCA with a computer science background, or equivalent experience.
- At least 7 years of total experience and a minimum of 3 years of relevant Databricks experience.
- Hands-on experience designing and implementing scalable Databricks data lakehouse solutions.
- Strong expertise in Python and PySpark for enterprise data pipelines and distributed processing.
- Extensive Snowflake experience, including data modeling, performance tuning, query optimization, cost management, Snowpipe, Streams, Tasks, and Data Sharing.
- Experience with Apache Iceberg, Delta Lake, schema evolution, partitioning, and time-travel capabilities.
- Strong understanding of AWS, Azure, or GCP cloud services, architecture, deployment, and platform operations.
- Experience building batch and real-time pipelines using Spark, Kafka, Event Hubs, Kinesis, Pub/Sub, and cloud-native services.
- Expertise in ETL/ELT design, data integration, cloud migration, data modeling, and distributed data processing.
- Experience leading development teams and delivering data modernization programs.
- Good-to-have experience with Terraform, CloudFormation, Docker, Kubernetes, GitHub Actions, Jenkins, Azure DevOps, or GitLab.
- Exposure to AI/ML, MLOps, GenAI data platform integration, Data Mesh, Data Fabric, or enterprise data architectures is preferred.
- Databricks, Snowflake, or cloud platform certifications are preferred.
- Strong troubleshooting, analytical, stakeholder management, communication, and delivery planning skills.
Benefits
- Extensive learning and development opportunities.
- Innovation-focused culture with opportunities for creativity and freedom.
- Holistic well-being support.
- Diversity, equity, and inclusion initiatives.
- Rewards and recognition opportunities.
- Competitive health and medical insurance coverage.
- Best-in-class amenities and workspaces.
- Full-time employment in Bangalore or Gurugram, India.
Tech Stack
Apache KafkaApache SparkAWSAzureDatabricksDockerGitHub ActionsGoogle Cloud PlatformJenkinsKubernetesPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Elevance Health
Fueled by our bold purpose to improve the health of humanity, we are transforming from a traditional health benefits organization into a lifetime trusted health partner. Our nearly 100,000 associates serve more than 118 million people, at every stage of health. We address a full range of needs with an integrated whole health approach, powered by industry-leading capabilities and a digital platform for health. We believe that improving health for everyone is possible. It begins by redefining health, reimagining the health system, and strengthening our communities.