3 months ago
Noida, IndiaSenior
Responsibilities
- Build and maintain high-volume ETL/ELT pipelines across Hadoop and AWS data services.
- Develop distributed data-processing solutions using PySpark, Spark SQL, and serverless cloud patterns.
- Design reusable batch ingestion frameworks and orchestrate workflows using Airflow, Control-M, Step Functions, or event-driven triggers.
- Optimize data workflows through partitioning, bucketing, compression, and Parquet/ORC file formats.
- Define technical specifications, make scalable architecture decisions, and build robust software solutions.
- Implement secure coding, unit testing, automation, code reviews, and continuous improvements to code quality.
- Troubleshoot Spark performance issues, job failures, cluster bottlenecks, and production incidents.
- Support data governance, compliance, data modeling, lineage, data profiling, and end-to-end data-flow design.
- Collaborate with business stakeholders, QA, and cross-functional teams to deliver high-quality datasets and streamline releases.
- Provide guidance to peers and junior team members and support business-as-usual and year-end activities when needed.
Requirements
- 4 to 8 years of overall ETL experience.
- Strong experience with Big Data technologies, Spark, Cloudera, AWS data services, and distributed computing.
- Hands-on expertise with Scala, PySpark, Spark optimization, HiveQL, SQL, and Impala.
- Experience with Hadoop ecosystem technologies including HDFS, Hive, Spark, YARN, and Kafka.
- Proficiency in Python or shell scripting.
- Strong experience with GitHub and Git commands.
- Experience with ETL, data warehousing, cloud concepts, data modeling, partitioning strategies, and schema evolution.
- Hands-on experience with Airflow, Control-M, or other workflow orchestrators.
- Understanding of serverless patterns, containerization, data security, and cloud compliance.
- Strong logical, analytical, problem-solving, communication, and stakeholder collaboration skills.
- AWS Data Engineer or Developer certification is preferred.
Benefits
- Health benefits beginning on Day One, wellbeing programs, retirement plans, career growth opportunities, continuing education and training, and a competitive total rewards package.
- Virtual interviews are conducted by video, with a camera-on culture and possible occasional travel to a physical office location.
- Employment is contingent on a background check, which may include identity, education, employment, criminal, sanctions, credit, and drug-test checks depending on the position.
Tech Stack
Amazon RedshiftApache AirflowApache HadoopApache HiveApache KafkaApache SparkAWSDockerGitPythonScalaSQLYarn
Categories
Data Engineering
About Alight
Alight provides cloud-based benefits administration and human capital solutions to large employers, combining its Alight Worklife platform with outsourced services for health, wealth, leave, and wellbeing programs. It earns revenue through SaaS subscriptions and long-term managed services and consulting. Headquartered in Deerfield, Illinois, Alight was carved out of Aon’s benefits outsourcing business by Blackstone in 2017 and is now publicly traded on the NYSE (ticker: ALIT).