Lead Software Engineer (Data Engineer)
S&P Global1 day ago
Noida, India +2 moreStaff+
Responsibilities
- Assist with onboarding new data assets to the enterprise platform.
- Monitor, troubleshoot, maintain, and enhance existing data pipelines.
- Perform data validation, quality checks, metadata tagging, cataloging, and lineage documentation.
- Support data storage and processing using Amazon S3 and AWS Glue Data Catalog.
- Write and maintain Python scripts for data ingestion, processing, and automation.
- Support deployment, testing, monitoring, and pipeline health escalation.
- Contribute to deployment and operational workflow automation using AWS CloudFormation or similar tools.
- Participate in production incident response and troubleshooting.
- Document onboarding procedures, pipeline configurations, and operational processes.
- Collaborate with senior engineers and communicate progress and issues to stakeholders.
Requirements
- 5+ years of experience in data engineering, data support, or related roles.
- Basic knowledge of SQL, Python, and big data processing engines.
- Hands-on Python or similar programming experience for data processing and automation.
- Familiarity with cloud data platforms such as Databricks or similar.
- Understanding of data onboarding, data quality, metadata, and data lineage concepts.
- Hands-on experience with AWS services including Amazon S3, AWS Glue Data Catalog, AWS IAM, and AWS CloudWatch.
- Strong problem-solving, attention to detail, and collaboration skills.
- Preferred: exposure to data pipelines, data onboarding, or data migration projects.
- Preferred: knowledge of Delta Lake, Iceberg, or similar storage formats.
- Preferred: understanding of data governance and security principles.
- Preferred: relevant data engineering or AWS certifications or training.
Benefits
- Health and wellness coverage.
- Generous paid time off and flexible downtime.
- Continuous learning resources and career development opportunities.
- Competitive pay, retirement planning, continuing education support, company-matched student loan contributions, and financial wellness programs.
- Family-friendly benefits and perks.
- Retail discounts and referral incentive awards.
- Role is located in Chennai, India.