2 months ago
Pune, India +2 moreMid Level
Responsibilities
- Build and maintain data pipelines ingesting data from third-party APIs and internal sources into cloud data lake and lakehouse environments.
- Develop Spark/PySpark and SQL transformations to standardize, model, enrich, and serve analytics-ready datasets.
- Build, schedule, monitor, and troubleshoot data pipeline orchestration workflows.
- Support data governance, cataloging, access control, and secure handling of sensitive multi-tenant data.
- Implement platform features including connector frameworks, taxonomy and rules engines, and data export capabilities.
- Integrate pipelines with Power BI, Tableau, and other visualization and reporting tools.
- Document architecture using C4 model diagrams and participate in design reviews.
- Troubleshoot data quality, pipeline failures, performance issues, and production incidents.
- Collaborate with DevOps and security teams on service accounts, credentials, infrastructure migrations, and containerization.
- Participate in on-call and support rotations for production data pipelines.
Requirements
- At least 3 years of experience as a Data Engineer building production-grade data pipelines.
- Hands-on experience with Apache Spark, PySpark, and SQL for data transformation at scale.
- Experience with cloud platforms, preferably Azure, and cloud-native data storage such as Data Lake or Blob Storage.
- Experience with Databricks and familiarity with Unity Catalog or similar governance and catalog tools.
- Familiarity with RBAC/ABAC data governance and access control models for sensitive, multi-tenant data.
- Experience integrating data pipelines with Power BI, Tableau, or similar BI and visualization tools.
- Comfort with API-based data ingestion tools or connectors such as Adverity is a plus.
- Understanding of software engineering practices including version control, testing, code review, and CI/CD.
- Strong communication and cross-functional collaboration skills.
- Experience with Apache Airflow, Okta, Entra ID, Istio, AKS/Kubernetes, PlantUML, or C4 model architecture documentation is preferred.
- Exposure to sustainability, ESG, or carbon accounting data models is preferred.
Benefits
- Full-time, permanent position.
- Location: DGS India, Bengaluru, Manyata N1 Block.
- Participation in on-call and production support rotations as needed.
Tech Stack
Categories
Data Engineering
About Dentsu
Dentsu is a global advertising and marketing services group that plans and buys media, develops creative campaigns, and builds data- and technology-led customer experiences for brands. It earns revenue from agency fees and media services across a network that includes Carat, iProspect, and Merkle. Founded in 1901 and headquartered in Tokyo, Dentsu is publicly traded in Japan and serves clients across many sectors worldwide.