Lead Data Engineer
Cohere Health5 months ago
Hyderābād, IndiaStaff+
Responsibilities
- Lead complex data engineering initiatives across multiple teams from design through production deployment.
- Design and implement scalable batch and near-real-time data pipelines.
- Build reusable ingestion, transformation, validation, and publishing frameworks.
- Support modernization of legacy workloads and lead Apache Iceberg lakehouse implementation and optimization.
- Define standards for schema evolution, partitioning, compaction, and metadata management.
- Implement data quality frameworks and improve observability, monitoring, operational excellence, and reliability.
- Review architecture proposals, design documents, code, and engineering practices.
- Mentor engineers across levels and establish technical best practices.
- Collaborate with Staff Engineers and Managers on platform architecture, technical strategy, and system reliability goals.
- Translate requirements across data platform engineering, analytics, product, and business teams into scalable solutions.
- Assist with information security implementation and maintenance, security policy enforcement, team competency, and issue reporting.
Requirements
- 8–12 years of data engineering experience.
- Experience with healthcare claims and authorization datasets in insurance.
- Experience leading enterprise-scale data initiatives.
- Experience with AWS S3, Glue, Athena, Lake Formation, IAM, and security controls.
- Experience with Apache Iceberg, Delta Lake or Hudi, dbt, and Airflow or Step Functions.
- Proficiency in Python, SQL, and PySpark or Spark.
- Knowledge of information security practices.
- Preferred experience with Kafka or AWS MSK, SageMaker, Atlan or similar data catalogs, multi-tenant data platforms, data contracts, and Data Mesh concepts.
- Ability to reliably commute to or relocate to Hyderabad, Telangana before starting.
Tech Stack
Categories
Data Engineering
About Cohere Health
Cohere Health builds a clinical intelligence and operations platform used by U.S. health plans and provider organizations to streamline prior authorization and align care with evidence-based policies. Its SaaS products connect payers and providers across utilization management, policy management, and payment accuracy workflows, reducing administrative burden and speeding decisions. Founded in 2019, the privately held company is headquartered remotely and sells enterprise contracts to national and regional health plans.