07.Data Engineer
PayPay India2 months ago
Gurgaon, IndiaSenior
Responsibilities
- Create and manage data ingestion pipelines using Databricks, Airflow, Kafka, and Terraform.
- Optimize large-scale data pipelines for performance, reliability, and efficiency.
- Develop data processing workflows with Databricks, Delta Lake, and Spark.
- Maintain and improve the data lakehouse using Unity Catalog for data management and discovery.
- Build automation, frameworks, and tools to streamline data engineering workflows.
- Collaborate across teams to support data flow and integration.
- Apply practices for observability, data governance, security, and regulatory compliance.
Requirements
- At least 7 years of experience as a Data Engineer or in a similar role.
- Hands-on experience with Databricks, Delta Lake, Spark, and Scala.
- Experience designing, building, and operating data lakes or data warehouses.
- Proficiency with data orchestration tools such as Airflow, Dagster, or Prefect.
- Familiarity with change data capture tools such as Canal, Debezium, or Maxwell.
- Strong command of Scala, Python, or another primary programming language and SQL.
- Experience with data catalog and metadata management tools including Unity Catalog or Lake Formation.
- Experience with infrastructure as code using Terraform.
- Strong problem-solving, debugging, communication, collaboration, decision-making, and learning abilities.
- Ability to use AI/LLM-based tools in development, reviews, testing, debugging, and documentation while maintaining human oversight and accountability.
Tech Stack
Categories
Data Engineering
About PayPay India
PayPay India develops software for PayPay Corporation’s smartphone payment platform in Japan, covering QR-code payments and related fintech services for consumers and merchants. Established in 2022 in Gurugram as PayPay’s second development center, it collaborates with teams in Japan and Canada and draws on technical ties with Paytm. The platform serves a nationwide user base in Japan and monetizes primarily through merchant acceptance fees and financial services.