Illumina

Senior Data Engineer

Illumina
Apply
3 days ago
Bengaluru, IndiaSenior

Responsibilities

  • Own end-to-end onboarding of assigned SAP objects, including source analysis, CDS wrapper view creation, Datasphere replication setup, Bronze ingestion, validation, and sign-off.
  • Design, build, and maintain Databricks Bronze pipelines using Auto Loader, Lakeflow Declarative Pipelines, PySpark, and Delta Lake for full, delta, CDC, and watermark loads.
  • Operate and monitor Datasphere replication flows into Amazon S3 and recover failed runs with platform administration support.
  • Perform source-to-Bronze reconciliation, investigate aggregate and record-level discrepancies, and document corrective actions.
  • Plan and execute SIT/UAT scenarios covering loads, deletes, schema changes, SAP validation, and acceptance evidence.
  • Monitor observability dashboards and alerts, resolve ingestion failures and schema drift, lead root cause analysis, and implement preventive actions.
  • Promote changes through Git and CI/CD, review code and configurations, and apply Unity Catalog access controls.
  • Develop reusable automation for onboarding, reconciliation, monitoring, and recovery.
  • Maintain SAP object inventories, operational documentation, recovery procedures, and knowledge-transfer materials.
  • Resolve Jira and ServiceNow tickets, document hypercare incidents, mentor junior engineers, and participate in rotating after-hours on-call support.

Requirements

  • At least 6 years of enterprise data engineering or data integration experience with hands-on Databricks delivery and production SAP data integration, or the stated education-and-experience alternatives.
  • Hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Auto Loader, Lakeflow Declarative Pipelines, Workflows, Databricks SQL, and Unity Catalog.
  • Advanced SQL and Python skills for reconciliation, validation, aggregate comparisons, and automation.
  • Knowledge of SAP S/4HANA table structures, CDS view development, ODP delta extraction, and SE16 record-level verification.
  • Understanding of full, delta, pseudo-delta, and CDC loads, including watermarks, deletes, restartability, error handling, and schema drift.
  • Experience with Git, code review, testing, and CI/CD deployments across environments.
  • Experience with monitoring and incident management using Databricks dashboards, Grafana, Jira, and ServiceNow.
  • Ability to troubleshoot complex production issues, lead root cause analysis, maintain documentation, communicate across teams, and mentor junior engineers.
  • Willingness to work occasional non-standard hours, participate in rotating on-call coverage, and potentially travel internationally.
  • Preferred experience includes SAP Datasphere or Business Data Cloud replication flows, SLT, HVR, Fivetran, ERP-program SIT/UAT, cutover and hypercare, dbt, and AI-enabled engineering tools such as Cursor, Claude, ChatGPT, or GitHub Copilot.

Benefits

  • Inclusive workplace with employee resource groups, career development opportunities, cultural awareness initiatives, and social responsibility engagement.
  • Illumina states a zero-net pay gap regardless of gender, ethnicity, or race.
  • Equal opportunity employment and accommodation support are provided.
  • The role may require international travel, occasional non-standard hours, and rotating on-call coverage.

Tech Stack

DatabricksdbtGitGrafanaPython

Categories

Data Engineering
Illumina

About Illumina

5,001-10,000 employees

Illumina builds next-generation DNA sequencing systems, reagents/consumables, and bioinformatics software used by research, clinical, pharmaceutical, and agricultural genomics labs. Its business model combines sale of instruments with recurring consumables, sequencing services, and cloud analysis tools; the company was founded in 1998, is headquartered in San Diego, and trades on NASDAQ under the ticker ILMN.

Contact me