1 day ago
Chennai, IndiaStaff+
Responsibilities
- Design, build, and operate large-scale batch and streaming data pipelines using Spark, Java, and Python.
- Apply software engineering, architecture, testing, code review, and design practices to data engineering systems.
- Build and maintain services, APIs, and distributed systems supporting data pipelines and platform capabilities.
- Integrate embeddings, vector search, LLM-assisted data quality, anomaly detection, agentic monitoring, self-healing, and AI-assisted documentation into the data platform.
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure.
- Own data quality, observability, governance, reliability metrics, SLO/SLA practices, incident management, and post-incident reviews.
- Lead architecture and design reviews across data engineering and platform engineering domains.
- Set technical direction and best practices across multiple teams, mentor engineers, and influence broader data engineering standards.
- Evaluate and adopt Big Data, cloud, AI, and agentic technologies where they improve reliability, productivity, or business value.
Requirements
- 12+ years of professional software engineering experience with significant hands-on data engineering at scale.
- Strong proficiency in Java and Python and strong working knowledge of Spark for distributed data processing.
- Deep experience with Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow, batch processing, streaming processing, and large-scale data transformation.
- Experience with data modeling, ETL/ELT pipeline design, data quality, observability, governance, and lakehouse architectures such as Delta Lake, BigQuery, or Databricks.
- Hands-on experience integrating model APIs and GenAI/LLM tooling, building agentic workflows, working with embeddings and vector search, and applying MLOps practices such as feature stores, model lifecycle management, and retraining pipelines.
- Working knowledge of platform engineering, microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices.
- Strong distributed systems, event-driven architecture, asynchronous processing, fault tolerance, domain-oriented design, low-level design, system design, and production trade-off evaluation skills.
- Strong software engineering craftsmanship, including testing strategy, code review standards, design patterns, and high-throughput system design.
- Demonstrated ability to translate ambiguous data or business problems into technical execution plans and operate with high autonomy in a 0-to-1 environment.
- Additional experience with Go, C++, or Scala is preferred.
- A bachelor's or master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience; Walmart's minimum qualifications also allow qualifying experience in place of a degree.
Benefits
- Performance incentive awards, maternity and parental leave, paid time off, health benefits, and other benefits.
- Role is based in the Bangalore or Chennai office and requires daily work there, with flexibility to manage personal life needs.
Tech Stack
Apache AirflowApache HadoopApache HiveApache KafkaApache SparkC++DatabricksGoGoogle BigQueryJavaKubernetesPythonScala
Categories
BackendData Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.