4 days ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, and operate batch and streaming data pipelines using Spark, Java, and Python.
- Apply software engineering practices including clean architecture, automated testing, CI/CD, code reviews, and design patterns to data engineering.
- Evaluate dependencies, trade-offs, failure modes, scalability, and cost through system design and low-level design.
- Contribute to platform services, APIs, and distributed systems supporting data pipelines.
- Apply embeddings, vector search, LLM-assisted data quality and anomaly detection, agentic monitoring, and AI-assisted schema and lineage documentation.
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure.
- Own data quality, observability, governance, reliability metrics, SLOs, SLAs, and incident-management practices.
- Participate in architecture and design reviews across data and platform engineering.
- Provide technical judgment in design and code reviews, mentor early-career engineers, and improve engineering quality.
- Propose practical uses for emerging Big Data, cloud, AI, and agentic technologies.
Requirements
- At least 5 years of professional software engineering experience with hands-on data engineering at scale, or the stated bachelor's/master's degree and experience alternatives.
- At least 2 years of experience in data engineering, database engineering, business intelligence, or business analytics.
- Strong proficiency in Java and Python and strong working knowledge of Spark are required.
- Experience with Spark, Hadoop/HDFS, Hive, Kafka, Airflow or equivalent workflow orchestration, batch processing, streaming processing, and large-scale data transformation.
- Experience with data modeling, ETL/ELT pipeline design, data quality, observability, governance, and lakehouse architectures such as Delta Lake, BigQuery, or Databricks.
- Hands-on experience integrating model APIs and GenAI/LLM tooling, building agentic workflows, using embeddings and vector search, and applying MLOps practices.
- Working knowledge of microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices.
- Strong distributed systems, event-driven architecture, asynchronous processing, fault tolerance, system design, and low-level design skills.
- Strong software engineering craftsmanship, ownership, autonomy, technical judgment, and ability to translate data or business problems into execution plans.
- Experience with Go, C++, or Scala is preferred.
Benefits
- Maternity and parental leave, paid time off, health benefits, incentive awards, and other benefits.
- Based in the Bangalore or Chennai office for daily work, with flexibility to manage personal lives.
Tech Stack
Apache AirflowApache HadoopApache HiveApache KafkaApache SparkC++DatabricksGoGoogle BigQueryJavaKubernetesPythonScala
Categories
Data Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.