Lead Software Engineer (Data Engineer)
S&P Global1 day ago
Noida, India +3 moreStaff+
Responsibilities
- Assist with onboarding new data assets to the enterprise data platform.
- Monitor, troubleshoot, maintain, and enhance existing data pipelines.
- Support data ingestion, transformation, storage, processing, validation, quality checks, metadata management, and lineage documentation.
- Use Amazon S3 and AWS Glue Data Catalog to support cloud storage, data cataloging, discoverability, and governance.
- Write and maintain Python scripts for data processing, ingestion, and automation.
- Support deployment, testing, monitoring, incident response, and operational troubleshooting.
- Contribute to deployment and operational workflow automation using AWS CloudFormation or similar tools.
- Document onboarding procedures, pipeline configurations, and operational processes while collaborating with senior engineers and stakeholders.
Requirements
- 5+ years of experience in data engineering, data support, or related roles.
- Basic knowledge of SQL, Python, and big data processing engines.
- Hands-on programming experience in Python or similar languages for data processing and automation.
- Familiarity with cloud data platforms such as Databricks or similar technologies.
- Understanding of data onboarding, data quality, metadata, and data lineage concepts.
- Hands-on experience with AWS services including Amazon S3, AWS Glue Data Catalog, AWS IAM, and AWS CloudWatch.
- Preferred: exposure to data pipelines, data onboarding, or data migration projects.
- Preferred: knowledge of Delta Lake, Iceberg, or similar storage formats.
- Preferred: basic understanding of data governance and security principles and relevant data engineering or AWS certifications or training.
- Ability to solve problems, pay attention to detail, and collaborate effectively.
Benefits
- Health care coverage and wellness benefits.
- Generous paid time off and flexible downtime.
- Continuous learning resources and career development support.
- Retirement planning, continuing education support, company-matched student loan contributions, and financial wellness programs.
- Family-friendly benefits and perks.
- Retail discounts and referral incentive awards.