Senior GCP Data Engineer (R-19655)
Dun & Bradstreet4 months ago
Hyderābād, IndiaSenior
Responsibilities
- Design, build, and optimize scalable data pipelines and ETL/ELT workflows for large, complex datasets.
- Design and implement data architecture supporting identity resolution and ID graph systems.
- Develop ingestion, normalization, matching, and deduplication systems for identity graph construction.
- Process and unify multi-source datasets including cookies, device IDs, behavioral data, third-party data, and proprietary data.
- Write efficient, testable, and maintainable Python and SQL code for large-scale data processing.
- Optimize data models, queries, and storage strategies for performance, scalability, and cost efficiency.
- Build and maintain data validation, monitoring, and alerting systems for data quality and reliability.
- Troubleshoot, debug, and improve existing data pipelines and infrastructure.
- Own complex data problems from initial design through production deployment.
- Contribute to technical decisions involving data architecture, scalability, and system design.
- Collaborate with data, platform, DevOps, and product teams to deliver production-ready solutions.
- Translate business and product requirements into practical data solutions and document pipelines, systems, and workflows.
- Continuously improve system performance, data quality, and pipeline resilience.
Requirements
- 5–8+ years of hands-on experience in data engineering or large-scale data processing.
- Experience building and maintaining production-grade data pipelines and distributed systems.
- Experience contributing to the architecture and delivery of large-scale data platforms or mission-critical data systems.
- Strong SQL and relational database expertise, including experience with systems such as Postgres, BigQuery, or Redshift.
- Strong Python experience for data processing and analysis.
- Experience with Google Cloud Platform services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Cloud Functions, and/or AWS services such as S3, Redshift, EMR, and RDS.
- Experience working with datasets ranging from hundreds of millions to billions of records.
- Strong understanding of data modeling, partitioning, indexing, and query optimization.
- Experience with distributed data processing, parallelization, and moving large data volumes across systems and architectures.
- Familiarity with Docker, Kubernetes, and GitHub Actions.
- Strong debugging and troubleshooting skills in complex data environments.
- Experience with Git and tools such as Jira and Confluence.
- Strong analytical ability, attention to detail, independent execution, and system design skills.
Benefits
- Remote work arrangement in a fast-paced, collaborative environment.
Tech Stack
Amazon RedshiftAWSDockerGitGitHub ActionsGoogle BigQueryGoogle Cloud PlatformKubernetesPostgreSQLPythonSQL
Categories
Data Engineering
About Dun & Bradstreet
Dun & Bradstreet builds data and analytics products that help companies assess credit, manage third‑party risk and compliance, and find customers. Its Data Cloud, business credit reports, D‑U‑N‑S Number identity, and APIs are sold as subscriptions and software to finance, procurement, and go‑to‑market teams. Founded in 1841, the company is headquartered in Jacksonville, Florida, and is publicly traded on the NYSE (ticker: DNB).