15 days ago
Lucknow, IndiaMid Level
Responsibilities
- Develop, maintain, and optimize end-to-end batch and streaming data pipelines for advertising and event data.
- Build ingestion and transformation workflows using Python, SQL, Spark/PySpark, and data engineering frameworks.
- Implement data quality checks, schema validation, error handling, monitoring, retries, checkpointing, and recovery mechanisms.
- Work with distributed processing and streaming technologies including Apache Spark, Apache Kafka, Apache Flink, and Spark Structured Streaming.
- Optimize SQL queries, data models, partitioning strategies, table structures, compute usage, storage usage, and pipeline performance.
- Contribute to data lakehouse ingestion, storage, transformation, serving, metadata management, access controls, and retention practices.
- Work with analytical data stores and lake table formats such as StarRocks, ClickHouse, Apache Iceberg, Delta Lake, and Apache Hudi.
- Use workflow orchestration tools to schedule, monitor, and manage data pipelines.
- Investigate production issues involving pipeline failures, performance bottlenecks, data quality discrepancies, and reliability.
- Collaborate with backend engineers, ML engineers, product stakeholders, and other teams to deliver reliable data solutions.
Requirements
- 2–5 years of experience in software engineering, data engineering, or big data development.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.
- Production experience developing and maintaining data pipelines.
- Strong programming skills in Python and good command of SQL.
- Hands-on experience with Apache Spark/PySpark or a comparable distributed data processing framework.
- Experience building and maintaining ETL/ELT pipelines and understanding of batch and streaming data concepts.
- Exposure to workflow orchestration tools such as Apache Airflow, Prefect, or Dagster.
- Understanding of data modeling, partitioning, schema evolution, data quality, monitoring, logging, error handling, and production troubleshooting.
- Familiarity with Apache Kafka or similar messaging and streaming technologies and exposure to AWS, GCP, or Azure data services.
- Familiarity with data lakes, lakehouse architectures, or analytical databases.
- Preferred experience includes high-volume or near-real-time event pipelines, ad-tech datasets, Java or Scala, REST APIs, microservices, Docker, Kubernetes, CI/CD tools, and ML feature engineering or training data pipelines.
Benefits
- InMobi describes continuous learning and career progression through its Live Your Potential program.
- InMobi is an equal employment opportunity employer and provides reasonable accommodations to qualified applicants and employees with disabilities.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkAWSAzureClickHouseDockerGoogle Cloud PlatformJavaKubernetesPythonScalaSQL
Categories
Data Engineering
About InMobi
InMobi builds a mobile advertising and consumer engagement platform for advertisers and publishers and operates Glance, a lock‑screen content and commerce service. It sells programmatic ad buying, monetization tools, and AI-driven personalization across apps, TVs, and the open web. Founded in 2007 and headquartered in Singapore, its ad network reaches over 2 billion people in 150+ countries and serves more than 30,000 brands.