1 month ago
Chennai, India +2 moreSenior / Staff+
Responsibilities
- Design, build, and support distributed systems and large-scale data processing systems in production.
- Build batch and streaming data pipelines on Google Cloud.
- Work with operational data stores, data warehouses, big data technologies, and data lakes.
- Develop complex queries and data-processing solutions using Python, SQL, Java, Scala, or Go.
- Use distributed frameworks such as Spark, Flink, Beam, and Hadoop.
- Work with containers and cloud platforms including GCP, Azure, and AWS.
- Implement real-time streaming solutions using Kafka technologies.
- Support automation and deployment workflows using Jenkins, GitHub, and Spinnaker.
- Handle clients and lead a feature team to deliver outcomes.
Requirements
- 8–12 years of hands-on experience in GCP data engineering and production distributed systems.
- Expertise with GCP data products including BigQuery, Dataflow, Cloud Storage, Pub/Sub, Cloud Composer, and Apache Kafka.
- Strong commercial coding or scripting experience in Python, SQL, Java, Scala, or Go, including complex SQL query development.
- Experience with operational data stores, data warehouses, big data technologies, and data lakes.
- Experience with Spark, Flink, Beam, and Hadoop.
- Knowledge of Docker, Kubernetes, and cloud platforms such as GCP, Azure, or AWS.
- Experience with real-time streaming and Kafka technologies.
- Experience with Jenkins, GitHub, and Spinnaker for automation and deployment.
- Understanding of data structures, algorithms, software design, design patterns, and core programming concepts.
- Client-handling experience and the ability to lead a feature team.
- GCP Cloud Architect, Cloud Developer, or Professional Data Engineer certification is useful.
- Apache Kafka CCDAK certification is useful.
Benefits
- Work location options are Bangalore, Hyderabad, or Chennai.
- Work on projects for international and local banks, insurance companies, payment service providers, and other financial-services clients.
- Open culture emphasizing diversity, inclusion, creativity, and career growth without a forced hierarchy.
Tech Stack
Apache BeamApache FlinkApache HadoopApache KafkaApache SparkAWSAzureDockerGoGoogle BigQueryGoogle CloudGoogle Cloud PlatformJavaJenkinsKubernetesPythonScalaSpinnakerSQL
Categories
Data Engineering
About Capco
Capco is a global technology and management consultancy focused on financial services, helping banks, capital markets, insurance, and wealth firms with digital transformation, risk and compliance, and data/AI-enabled modernization. Founded in 1998 and headquartered in London, it operates as a Wipro company. Its business model centers on project-based consulting, managed services, and solution delivery across strategy, engineering, and platforms for regulated institutions.