S&P Global

Senior Data Engineer

S&P Global
Apply
2 days ago
Hyderābād, IndiaSenior

Responsibilities

  • Migrate and stabilize enterprise data pipelines onto Databricks and AWS lakehouse platforms.
  • Design and operate batch, streaming, event-driven, metadata-driven, and configuration-driven data pipelines.
  • Implement reusable ingestion, transformation, testing, deployment, monitoring, observability, and operational support patterns.
  • Apply AWS security controls, encryption, access policies, governance guardrails, lineage, metadata, and auditability practices.
  • Integrate pipelines and datasets with data mastering workflows, including cleansing, enrichment, reconciliation, golden-copy generation, and downstream distribution.
  • Use AI-assisted engineering tools and context-engineering practices for code generation, refactoring, testing, documentation, troubleshooting, and pipeline analysis.
  • Provide senior technical guidance and contribute to architecture discussions, design reviews, code reviews, standards, and production-readiness reviews.
  • Support global teams through reusable onboarding frameworks, technical documentation, runbooks, reference implementations, and engineering playbooks.

Requirements

  • 6–8+ years of experience in data engineering, data platforms, cloud data architecture, software engineering, or related engineering domains.
  • Experience working as a senior individual contributor, technical lead, senior data engineer, lead data platform engineer, or equivalent.
  • Hands-on experience building and operating modern batch and/or streaming data pipelines.
  • Experience with Databricks data platforms, lakehouse architecture, AWS-native data engineering, and production pipeline operations.
  • Experience with data quality, schema management, lineage, metadata, access control, encryption, governance, observability, and reliability practices.
  • Experience with data mastering, MDM, reference data, market data, investment data, risk data, or trusted data distribution workflows.
  • Practical experience using Claude, GitHub Copilot, LLMs, or similar AI-assisted development platforms, including prompt and context engineering.
  • Ability to evaluate and refine AI-generated code and recommendations while following security, privacy, compliance, and human-review requirements.
  • Ability to provide technical guidance without direct management authority and collaborate across global, offshore, architecture, governance, security, and business-aligned teams.
  • Preferred qualifications include enterprise data onboarding, semantic modeling, metadata and cataloging, NeoXam DataHub, financial-services mastering workflows, retrieval-augmented generation, embeddings, vector search, knowledge bases, relevant certifications, and regulated-environment experience.

Benefits

  • Health care coverage and wellness benefits.
  • Generous flexible downtime and paid time off.
  • Continuous learning resources and career development support.
  • Retirement planning, continuing education, company-matched student loan contributions, and financial wellness programs.
  • Family-friendly benefits and perks for partners and children.
  • Retail discounts and referral incentive awards.
  • Benefits vary by country; additional details are provided through the company benefits site.

Tech Stack

DatabricksSQL

Categories

Data Engineering
Contact me