3 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Design, build, test, and deploy scalable data solutions and distributed Spark systems.
- Develop and optimize ETL pipelines, data architectures, datasets, and transformation logic for terabyte- and petabyte-scale processing.
- Perform root cause analysis, failure analysis, rollback planning, and data/process optimization.
- Implement monitoring, logging, tracing, alerting, access controls, encryption, and data masking to support secure and highly available systems.
- Design data models and data warehousing solutions using distributed systems and large-scale data processing techniques.
- Collaborate with engineering, product, business, and geographically distributed teams to deliver technical solutions and product features.
- Research, evaluate, and pilot open-source tools and technologies for data engineering.
- Write integration, regression, performance, and unit tests and support release management.
- Mentor, coach, and support junior team members and contribute to engineering best practices.
Requirements
- Bachelor's degree in computer science, computer engineering, computer information systems, software engineering, or a related area and 4 years of software engineering experience, or 6 years of software engineering experience without the degree.
- The role lists an experience range of 9–14 years, including 11 years of hands-on Python object-oriented programming experience.
- At least 8 years of experience with database fundamentals, SQL, PL/SQL, and stored procedures.
- At least 7 years of experience designing and developing large-scale, fault-tolerant, highly scalable data pipelines using big data technologies.
- Strong knowledge of system design, algorithms, distributed systems, high-scale data processing, ETL, data modeling, and data warehousing.
- Experience with Databricks, Spark, Kafka, Cassandra, Hadoop, Hive, Presto, Airflow, and cloud infrastructure.
- Experience with configuration-driven, behavior-driven, and test-driven development, DevOps practices, and CI/CD tools.
- Experience building systems that handle failures, concurrency, data inconsistencies, high-volume traffic, and petabyte-scale processing.
- Experience mentoring junior resources and balancing performance, cost, and reliability.
- Master's degree in a relevant field plus 2 years of software engineering experience is preferred.
Benefits
- Performance incentive awards, maternity and parental leave, paid time off, health benefits, and other benefits are available.
- The role is based in the Bangalore or Chennai office for daily work, with flexibility to manage personal lives.
Tech Stack
Apache AirflowApache CassandraApache HadoopApache HiveApache KafkaApache SparkAWSAzureDatabricksGitHub ActionsGitLab CI/CDGoogle Cloud PlatformJenkinsOpenStackPrestoPythonScalaSQL
Categories
Data Engineering
About Walmart
Walmart is a global omnichannel retailer that sells groceries, general merchandise, and services through Walmart and Sam's Club stores and online marketplaces for consumers and small businesses. Its model spans physical retail, e-commerce, advertising, and a vast supply chain, plus pharmacy, health, and financial services. Founded in 1962 and headquartered in Bentonville, Arkansas, Walmart Inc. is a public company on the NYSE (WMT) operating in the U.S. and multiple international markets.