30 days ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, and operate batch and streaming data pipelines using Spark, Java, and Python at Walmart scale.
- Apply software engineering, automated testing, CI/CD, code review, system design, and low-level design practices to data engineering work.
- Contribute to platform services, APIs, and distributed systems supporting data pipelines.
- Apply AI, GenAI, embeddings, vector search, LLM-assisted quality detection, agentic monitoring, and AI-assisted schema and lineage documentation to data platforms.
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure.
- Own data quality, observability, governance, reliability metrics, incident management, and operational excellence for pipelines and datasets.
- Participate in architecture and design reviews, mentor early-career engineers, and raise engineering quality standards.
- Evaluate emerging Big Data, cloud, AI, and agentic technologies for reliability, productivity, and business value.
Requirements
- 5+ years of professional software engineering experience with hands-on data engineering at scale is required; strong Java, Python, and Spark proficiency is required.
- Experience with Spark, Hadoop/HDFS, Hive, Kafka, Airflow or equivalent workflow orchestration, batch processing, streaming processing, and large-scale data transformation.
- Experience with data modeling, ETL/ELT pipeline design, data quality, observability, governance, and lakehouse architectures such as Delta Lake, BigQuery, or Databricks.
- Hands-on experience integrating model APIs and GenAI/LLM tooling, building agentic workflows, using embeddings/vector search, and applying MLOps practices such as feature stores and retraining pipelines.
- Working knowledge of microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices.
- Strong distributed-systems, event-driven, asynchronous-processing, fault-tolerance, system-design, and low-level-design skills.
- Strong software engineering craftsmanship, including testing strategy, code review standards, design patterns, and high-throughput system design.
- Ability to translate data and business problems into technical execution plans and operate autonomously in a newly formed 0-to-1 team.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
- Additional experience with Go, C++, or Scala is preferred.
Benefits
- Performance incentive awards, maternity and parental leave, PTO, health benefits, and other benefits are available.
- The role is based in the Bangalore or Chennai office and requires daily work from the office, with flexibility to manage personal lives.
Tech Stack
Apache AirflowApache HadoopApache HiveApache KafkaApache SparkC++DatabricksGoGoogle BigQueryJavaKubernetesPythonScala
Categories
Data Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.