Lead Data Engineer
London Stock Exchange Group8 days ago
Hyderābād, IndiaStaff+
Responsibilities
- Build data ingestion pipelines from telemetry systems and curate usage datasets in Snowflake.
- Implement rating logic, billing-ready datasets, and SAP invoice feed generation.
- Develop data quality, reconciliation, audit control, exception-handling, and error-management processes.
- Build and maintain CI/CD pipelines for data engineering deployments.
- Implement monitoring, alerting, operational dashboards, technical documentation, and runbooks.
- Support batch and event-driven ingestion, data modeling, performance tuning, cost optimization, and troubleshooting.
- Collaborate with Finance, Engineering, and Analytics teams and handle QA and minor development tasks.
Requirements
- Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
- Experience with AWS Glue, PySpark, Python, SQL, cloud-based data processing, and Snowflake.
- Understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.
- Experience building data quality checks, reconciliation frameworks, audit controls, and exception-handling processes.
- Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.
- Strong SQL skills for analysis, validation, troubleshooting, and performance optimization.
- Experience with batch and event-driven data ingestion patterns, data modeling, partitioning, performance tuning, and cost optimization.
- Strong analytical, communication, collaboration, ownership, and attention-to-detail skills.
Benefits
- Blended hybrid work arrangement requiring attendance in the office at least three days per week, with flexibility to discuss work arrangements.
- Annual leave and parental leave.
- Job-relevant training, certifications, and ongoing professional development opportunities.
- Employee Assistance and wellbeing support, including healthcare, retirement planning, paid volunteering days, and wellbeing initiatives.
- Equal opportunity workplace with opportunities to work on high-impact financial data and analytics projects.
About London Stock Exchange Group
London Stock Exchange Group (LSEG) provides market infrastructure, financial data, and analytics to banks, asset managers, and corporations. Its businesses span capital markets (London Stock Exchange), post-trade clearing (LCH), and information services including FTSE Russell indices and the Refinitiv data platform, funded by transaction fees, subscriptions, and licensing. Headquartered in London and publicly listed on the LSE, the group serves global customers seeking trading, clearing, benchmarks, and enterprise data solutions.