5 hours ago
Bengaluru, IndiaSenior
Responsibilities
- Create, develop, and maintain end-to-end data pipelines using Ab Initio.
- Build scalable batch and streaming workloads with Apache Spark, PySpark, and Scala.
- Develop and orchestrate real-time and near-real-time ingestion flows using Apache NiFi.
- Optimize and tune ETL/ELT pipelines, Spark workloads, NiFi flows, and Ab Initio components.
- Enhance automated testing and deployments through CI/CD practices.
- Deliver stakeholder demonstrations, communicate technical solutions, and share knowledge.
- Continue learning through internal and external training.
Requirements
- Experience building Big Data solutions with Hadoop, GCP, or AWS.
- Strong skills in Scala, Java, Python, Spark, Terraform, Kafka, Kinesis, BigQuery, Dataflow, and SQL.
- Strong Ab Initio expertise, including GDE, PDL, Conduct>IT, Express>IT, EME, parallel processing, and component optimization.
- Advanced Apache Spark proficiency, including PySpark/Scala, SQL, streaming, and tuning.
- Hands-on Apache NiFi experience covering flow creation, tuning, error handling, and security.
- Practical knowledge of Infrastructure as Code and CI/CD strategies.
- Proficiency with Kubernetes and familiarity with Confluence, JIRA, and version control using Git or SVN.
- Ability to present technical issues and solutions clearly, with enthusiasm for learning new technologies.
Benefits
- Full-time role with flexible working options, including job share and reduced hours.
- Learning and development support, including internal and external training.
- Inclusive workplace with reasonable adjustments available during the selection process.
Tech Stack
Apache HadoopApache KafkaApache SparkAWSExpressGitGoogle BigQueryGoogle Cloud PlatformJavaKubernetesPythonScalaSQLSubversionTerraform
Categories
Data Engineering
About BT
BT Group builds and operates telecom networks and digital services for consumers, enterprises, and public‑sector clients, selling broadband, fixed-line voice, mobile (via EE), and managed network, cloud, and cyber security services. It is headquartered in London and listed on the London Stock Exchange, with American depositary shares on the NYSE. BT also owns Openreach, which manages the UK’s fibre and copper access network used by multiple retail providers.