Principal Data Engineer
Lloyds Banking Group1 hour ago
Hyderābād, IndiaStaff+
Responsibilities
- Own delivery and operations for data products and platforms across multiple squads, with accountability for outcomes, quality, reliability, and production SLAs.
- Define roadmaps, OKRs, release plans, estimates, scope, risks, and stakeholder communications.
- Build and lead high-performing teams through hiring, coaching, performance management, coding standards, working agreements, and succession planning.
- Establish reusable engineering frameworks and golden paths for ingestion, transformation, data quality, lineage, and observability.
- Define enterprise semantic models, KPI and metric definitions, conformed dimensions, metric stores, and query-ready BI views.
- Shape data platforms for AI and machine learning through ontologies, knowledge graphs, feature and embedding strategies, and AI-ready patterns.
- Guide dimensional, star/snowflake, and Data Vault 2.0 modeling and establish cloud warehouse performance standards.
- Oversee Kafka, Pub/Sub, Kinesis, Flink, and Spark event pipelines with replay, exactly-once semantics, resiliency, and service-level objectives.
- Implement data contracts, quality rules, reconciliation, metadata capture, end-to-end lineage, telemetry, cost controls, IAM, encryption, secrets management, and auditability.
- Govern partner delivery and ensure reusable intellectual property is contributed to inner-source repositories.
Requirements
- 15+ years of experience owning complex data platform deliveries, leading multiple squads, and managing production outcomes and SLAs.
- Hands-on experience defining semantic layers, KPI and metric logic, metric stores, and BI consumption models.
- Practical understanding of AI and ML data needs, including feature engineering, knowledge graphs, ontologies, vector and embedding patterns, and operationalization on cloud platforms.
- Deep experience with dimensional modeling, star/snowflake schemas, Data Vault 2.0, slowly changing dimensions, and schema evolution or versioning.
- Strong experience with one or more of BigQuery, Snowflake, Redshift, Synapse, or Databricks SQL, including cloud data warehouse performance trade-offs.
- Experience with Kafka, Pub/Sub, Kinesis, Spark, or Flink, plus event schema management using Avro or Protobuf, idempotency, back-pressure, and state management.
- Software engineering experience with Git-based workflows, code reviews, automated testing, CI/CD, artifact versioning, documentation, and inner-source practices.
- Experience with Terraform or CloudFormation, Docker, Kubernetes, autoscaling, job concurrency, and runtime reliability.
- Knowledge of data contracts, Great Expectations or dbt tests, catalog and lineage tooling, and policy enforcement through pipelines.
- Experience with cloud data warehouse FinOps, workload tuning, budgets, alerts, platform telemetry, and actionable SLOs.
Benefits
- Hybrid working arrangement in Hyderabad with at least 2 days per week in the office.
- Flexible working options are supported.
- Role end date is Tuesday 29 September 2026.
Tech Stack
Amazon RedshiftApache FlinkApache KafkaApache SparkDatabricksdbtDockerGitGoogle BigQueryKubernetesSnowflakeTerraform
Categories
Data Engineering