1 month ago
Hyderābād, IndiaSenior
Responsibilities
- Build and operate batch and incremental Auto Loader ingestion pipelines into managed Delta tables.
- Design metadata-driven pipeline generation so onboarding and retention changes are configuration-driven rather than custom-coded per dataset.
- Implement partition-based retention, rolling retention windows, and archive extraction processes.
- Build and maintain a metadata-driven routing-rules engine that generates filtered and masked downstream consumption views.
- Own Unity Catalog catalog, schema, storage-location, access-grant, lineage, and audit-trail governance.
- Manage Databricks Asset Bundles or equivalent infrastructure-as-code deployments across development, test, and production environments.
- Monitor pipeline health, checkpoint integrity, schema evolution, job SLAs, ingestion failures, and data-freshness breaches.
- Collaborate with source-system owners and downstream teams on data contracts, access requirements, routing rules, and documentation.
Requirements
- Bachelor ’s degree in Business Analytics, Computer Science, or Statistics, or a Master’s degree in Data Science is listed among the qualifications.
- Strong experience with Azure Databricks, Unity Catalog, PySpark, SQL, Structured Streaming, Auto Loader, MERGE patterns, and Delta Lake internals.
- Demonstrated experience designing or maintaining metadata- or configuration-driven code generation for SQL, views, jobs, or DDL.
- Experience with parameterized pipeline patterns, generated-artifact synchronization, versioning, and change auditability.
- Knowledge of partitioning strategies, retention management, archive processing, and production-scale data volumes.
- Familiarity with Azure fundamentals including ADLS Gen2, Azure Data Factory or equivalent orchestration, networking, private endpoints, and cost management.
- Experience with CI/CD for data pipelines, Azure DevOps or GitHub Actions, Git-based workflows, and preferably Databricks Asset Bundles or Terraform.
- Experience with pipeline monitoring, observability, data-quality checks, schema-drift detection, access control, lineage, retention compliance, and PII masking.
- Ability to work with multiple downstream consumers and communicate pipeline and governance logic to technical and non-technical stakeholders.
- Nice-to-have experience with Delta Live Tables, Databricks Streaming Tables, Snowflake, Iceberg, or multi-tenant data platforms.
- English proficiency at C2 level is specified.
- Listed certifications include Certified Data Management Professional, Databricks Certified Associate Developer for Apache Spark, Microsoft Certified: Azure Data Engineer Associate, and Oracle Database 12c Certified Implementation Specialist.
Benefits
- Regular employment with office-based work and flex-time scheduling in India.
- Opportunities for hands-on experience, mentorship, and world-class learning and career development.
- Opportunity to work on AI-powered transformation and global enterprise data challenges.
- Values-driven environment emphasizing integrity, inclusion, innovation, and collaboration.
About Genpact
Genpact is a global professional services and technology firm that runs and transforms mission-critical operations for large enterprises through consulting, managed services, and AI- and analytics-driven platforms. It serves industries such as banking, insurance, healthcare, and manufacturing/supply chain, modernizing finance, risk, customer service, and back-office processes. Headquartered in New York and listed on the NYSE (G), Genpact originated within GE before becoming an independent public company.