2 months ago
Lucknow, IndiaStaff+
Responsibilities
- Architect and develop a self-service data platform integrating Spark, Flink, Airflow, Iceberg, internal services, and cloud offerings.
- Build platform integrations that enable push-button data workflows for data engineers and analysts.
- Operate distributed systems processing petabytes of data daily across multi-region Kubernetes infrastructure.
- Optimize Spark and Flink compute utilization and query performance for large-scale batch and real-time streaming workloads.
- Build telemetry, data quality, validation, drift detection, and automated incident-response frameworks for 24/7 uptime.
- Implement data governance, cataloging, lineage tracking, access control, and SLA/SLO enforcement.
- Develop Kubernetes operators, multi-tenancy, autoscaling, and reproducible infrastructure deployments.
Requirements
- 7–10 years of experience building, optimizing, and operating production data platforms.
- Deep knowledge of data modeling, partitioning strategies, and query optimization.
- Experience with Spark, PySpark, Scala, Flink streaming, and performance tuning at petabyte scale.
- Experience with Iceberg, Polaris, schema evolution, time travel, Airflow DAGs, dependency management, and SLA monitoring.
- Experience with dbt modeling, testing, documentation, incremental builds, Great Expectations, validation frameworks, and drift detection.
- Knowledge of Velox, Gluten, Parquet, ORC, OpenMetadata, data lineage, governance, and access control.
- Experience developing Kubernetes platforms with Spark/Flink operators, Yunikorn, multi-tenancy, and autoscaling.
- Experience with GKE, GCS, hybrid cloud or on-premises architecture, Terraform, and Helm.
- Programming experience in Python, PySpark, and Scala for data pipelines and platform tooling.
- Preferred experience includes cloud data platform or control-plane development, Prometheus/Grafana, Loki, FireHydrant, Kafka, exactly-once semantics, backpressure handling, and data or cost optimization.
Benefits
- InMobi offers continuous learning and career progression through its Live Your Potential program.
- The company is an equal employment opportunity employer and provides reasonable accommodations to qualified individuals with disabilities.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkdbtGrafanaHelmKubernetesPrometheusPythonScalaTerraform
Categories
Data Engineering
About InMobi
InMobi builds a mobile advertising and consumer engagement platform for advertisers and publishers and operates Glance, a lock‑screen content and commerce service. It sells programmatic ad buying, monetization tools, and AI-driven personalization across apps, TVs, and the open web. Founded in 2007 and headquartered in Singapore, its ad network reaches over 2 billion people in 150+ countries and serves more than 30,000 brands.