6 months ago
Egypt (city not specified)Mid Level
Responsibilities
- Build and operate batch and streaming ingestion into layered BigQuery data warehouses using Airflow, Debezium CDC, Kafka, protobuf, Pub/Sub, and Dataflow.
- Integrate marketing, payment, storage, and third-party API data sources, including schema contracts, backfills, and reconciliation.
- Develop Python-based Airflow operators, connectors, ETL framework components, Kafka Connect on Strimzi, Cloud Functions, and external API integrations.
- Build GitHub-based CI/CD and change-management tooling for BigQuery, including SQL migration, validation, backup, and rollback workflows.
- Own pipeline reliability and correctness through idempotency, deduplication, late-data handling, backfills, replay, monitoring, alerting, and automated tests.
- Drive data governance, compliance, IAM, PII protection, cataloging, lineage, and disaster-recovery planning.
- Build internal data tools and platform services, including Streamlit apps, Slack bots, LLM-based agents, and MCP servers.
- Support analysts and business teams with data requests and contribute to system design and architecture.
Requirements
- Strong practical Python skills and experience writing clean, modular, tested code for services, tooling, and data pipelines.
- Solid software design skills, including object-oriented programming, modularity, and design patterns.
- Experience building and operating cloud services with CI/CD, containerization, monitoring, and alerting in GCP, AWS, or a similar environment.
- Familiarity with Kubernetes and Terraform.
- Hands-on experience with a cloud data warehouse such as BigQuery and strong SQL skills.
- Clear communication with non-engineering stakeholders and ability to support analysts and business teams.
- Demonstrated ownership of technologies or services and proactive contribution to team ideas.
- Advanced SQL experience, including complex queries, window functions, partitioning, clustering, and cost optimization, is preferred.
- Experience with CDC pipelines such as Debezium, including idempotency, schema evolution, backfills, and reconciliation, is preferred.
- Analytical data modeling experience with table grain, facts and dimensions, slowly changing dimensions, and metric definitions is preferred.
- Experience with stream processing frameworks such as Flink or Apache Beam/Dataflow is preferred.
- Exposure to ITGC/SOX compliance, Databricks Unity Catalog, OpenMetadata, or Dataplex is preferred.
- Interest in building LLM-based agents and AI tooling for data is preferred.
Benefits
- Access to mentoring, career consulting, and learning programs.
- International collaboration and participation in the Global Talent Exchange Program.
- Company-wide challenges, awards, sports activities, social impact, and volunteering projects.
- Co-financed language courses and internal speaking clubs.
- Final benefits may vary depending on location.
Tech Stack
Apache AirflowApache BeamApache FlinkApache KafkaAWSDatabricksFlywayGoogle BigQueryGoogle Cloud PlatformKubernetesPythonTerraform
Categories
Data Engineering
About inDrive
inDrive builds a global mobility and urban services marketplace best known for ride-hailing, with additional offerings in intercity rides, delivery, and financial services. The privately held company is headquartered in Mountain View, California, and operates across 48 countries, monetizing through fees on transactions in its app. Its platform emphasizes driver–rider price negotiation and has amassed hundreds of millions of downloads.