3 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Design, build, test, deploy, and optimize scalable data solutions for Walmart-scale workloads.
- Develop high-performance Spark and big-data solutions for extracting, transforming, and loading large-scale datasets.
- Build and optimize data pipelines, architectures, and datasets processing terabytes and petabytes of data.
- Implement monitoring, logging, tracing, alerting, rollback, failure-analysis, and root-cause-analysis mechanisms.
- Implement fine-grained access controls, encryption, and data-masking techniques to protect sensitive information.
- Write integration, regression, performance, and unit tests and analyze programs before launch.
- Collaborate with engineering, product, business, and data teams to define priorities, requirements, and technical best practices.
- Research and evaluate open-source tools through proofs of concept and pilot programs.
- Mentor, coach, and support junior and other team members.
- Improve system performance, scalability, reliability, automation, and cost efficiency.
Requirements
- Bachelor’s or master’s degree in computer science or a related technical field, or qualifying equivalent software-engineering experience under the stated minimum-qualification options.
- 12 years of hands-on object-oriented programming experience in Python.
- At least 8 years of experience with database fundamentals, SQL, PL/SQL, and stored procedures.
- At least 7 years of experience designing and developing large-scale, fault-tolerant data pipelines using technologies such as Spark, Kafka, Cassandra, Hadoop, Hive, Presto, and Airflow.
- Strong knowledge of system design, algorithms, distributed systems, high-scale data processing, ETL, data modeling, and data warehousing.
- Experience designing distributed systems that handle failures, high concurrency, data inconsistencies, edge cases, high-volume traffic, and petabyte-scale processing.
- Experience with end-to-end product development, configuration-driven development, behavior-driven development, test-driven development, and DevOps best practices.
- Experience mentoring junior resources and using a platform-driven approach.
- Experience with Jenkins, GitHub Actions, or GitLab CI/CD.
- Experience with cloud infrastructure such as AWS, GCP, or OpenStack.
- Experience with metadata-driven development, automation, optimization, failure analysis, and rollback strategies.
- Strong problem-solving, critical-thinking, analytical, communication, and technical-explanation skills.
Benefits
- Daily work is based in the Walmart Bangalore or Chennai office, with flexibility to manage personal lives.
- Benefits include performance incentive awards, maternity and parental leave, paid time off, health benefits, and other company benefits.
Tech Stack
Apache AirflowApache CassandraApache HadoopApache HiveApache KafkaApache SparkAWSAzureGitHub ActionsGitLab CI/CDGoogle Cloud PlatformJenkinsOpenStackPrestoPythonScalaSQL
Categories
Data Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.