1 day ago
Bengaluru, IndiaStaff+
Responsibilities
- Design, build, and operate large-scale batch and streaming data pipelines using Spark, Java, and Python.
- Apply software engineering practices including clean architecture, automated testing, CI/CD, code reviews, domain-oriented design, system design, and low-level design.
- Build and maintain services, APIs, and distributed systems supporting data pipelines and the broader platform codebase.
- Integrate embeddings, vector search, LLM-assisted data quality and anomaly detection, agentic monitoring, self-healing workflows, and AI-assisted schema and lineage documentation.
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure.
- Own data quality, observability, governance, reliability metrics, SLOs, SLAs, incident management, and post-incident reviews.
- Lead architecture and design reviews across data engineering and platform engineering.
- Set technical direction and best practices across multiple teams, mentor engineers, and influence data engineering standards.
- Evaluate emerging Big Data, cloud, and AI technologies and adopt them where they improve reliability, productivity, or business value.
Requirements
- At least 12 years of professional software engineering experience with significant hands-on data engineering at scale is requested in the role description.
- Strong proficiency in Java and Python and strong working knowledge of Spark are required; Go, C++, or Scala experience is a plus.
- Minimum qualifications allow either a bachelor's degree in a relevant technical field plus 5 years of software engineering experience or 7 years of software engineering experience.
- Deep experience with Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow, batch and streaming processing, and large-scale data transformation.
- Experience with data modeling, ETL/ELT pipeline design, data quality, observability, governance, lakehouse architectures, Delta Lake, BigQuery, and Databricks.
- Hands-on experience with model APIs, GenAI/LLM tooling, agentic workflows, embeddings, vector search, and MLOps practices such as feature stores, model lifecycle, and retraining pipelines.
- Working knowledge of microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices.
- Strong distributed-systems, event-driven, asynchronous-processing, fault-tolerance, domain-oriented design, low-level design, testing, code review, and system-design skills.
- Ability to translate ambiguous data and business problems into technical execution plans and operate autonomously in a 0-to-1 platform environment.
- A master's degree plus 3 years of software engineering experience is listed as a preferred qualification; accessibility knowledge including WCAG 2.2 AA is also preferred.
Benefits
- Benefits include incentive awards, maternity and parental leave, paid time off, health benefits, and other benefits.
- The role is based in the Bangalore or Chennai office and requires daily work from the office, with flexibility to manage personal lives.
Tech Stack
Apache AirflowApache HadoopApache HiveApache KafkaApache SparkC++DatabricksGoGoogle BigQueryJavaKubernetesPythonScala
Categories
BackendData Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.