1 month ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, and maintain scalable batch, streaming, and near-real-time pipelines for post-payment audit, payment integrity, supplier reconciliation, recovery operations, and finance analytics.
- Create curated datasets, semantic data models, feature-ready tables, and reusable data products from invoice, purchase order, receiving, supplier, claims, pricing, and payment systems.
- Develop ELT/ETL and streaming workflows using SQL, Python, PySpark, Apache Flink, DBT, orchestration tools, event streaming platforms, and cloud data platforms.
- Build CDC pipelines with event-time processing, late-arriving data handling, replayability, idempotency, backfills, and failure recovery.
- Design and manage lakehouse data products supporting ACID transactions, incremental reads, upserts, time travel, and auditability.
- Implement data quality checks, reconciliation controls, schema validation, lineage, monitoring, alerting, and observability practices.
- Partner with audit, finance, product, data science, analyst, and engineering stakeholders to translate requirements into production-grade solutions.
- Support anomaly detection, exception prioritization, audit rules, ML scoring, and GenAI workflows through source data, feature engineering pipelines, and operational datasets.
- Build automation and data services that help audit teams summarize evidence, triage exceptions, generate investigation context, and reduce manual review.
- Apply retrieval-ready document stores, embeddings pipelines, vector database integrations, metadata enrichment, and governed knowledge retrieval patterns.
- Optimize large-scale batch and streaming jobs for reliability, freshness, latency, state management, compute efficiency, and downstream usability.
- Investigate data discrepancies, pipeline failures, reconciliation mismatches, and source-system issues across large and imperfect datasets.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Information Technology, Data Engineering, Analytics, Mathematics, Statistics, or a related field with 6+ years of relevant experience; alternatively, a master’s degree with 4+ years or a PhD with 3+ years.
- Strong hands-on experience with SQL, Python, PySpark, Apache Flink or equivalent stream processing frameworks, and large-scale distributed data processing.
- Experience with DBT or equivalent transformation frameworks, orchestration tools such as Airflow, event streaming platforms such as Kafka, and cloud or enterprise data platforms such as BigQuery, Hive, or Spark.
- Experience designing, building, deploying, and supporting production batch and streaming pipelines, ELT/ETL workflows, data models, and reusable data products.
- Understanding of data modeling, partitioning, performance tuning, incremental processing, CDC, schema evolution, data contracts, metadata management, lineage, and lakehouse table formats.
- Experience implementing data quality checks, reconciliation logic, validation frameworks, observability, monitoring, alerting, and incident resolution.
- Experience with event-time processing, stateful transformations, checkpointing, watermarking, exactly-once or effectively-once processing, and replay or backfill strategies.
- Experience working with large structured and semi-structured datasets from multiple source systems and inconsistent business definitions.
- Ability to collaborate across Finance, Audit, Operations, Product, Data Science, and Engineering teams.
- Minimum qualifications also include either a bachelor’s degree with 3 years of software engineering or related experience, 5 years of software engineering or related experience, or a master’s degree with 1 year of such experience, plus 2 years of experience in data engineering, database engineering, business intelligence, or business analytics.
Benefits
- Performance incentive awards, maternity and parental leave, paid time off, health benefits, and other company benefits.
- Role is based in the Bangalore or Chennai office and requires daily work from the office, with flexibility for associates to manage personal lives.
Tech Stack
Categories
Data Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.