1 day ago
Pune, IndiaEntry Level
Responsibilities
- Build and maintain reliable, scalable data pipelines and data products on Databricks and AWS.
- Implement data ingestion, validation, transformation, data modelling, and data quality checks.
- Write tested software and integrate and quality-assure data while meeting privacy and compliance needs.
- Contribute to technical design, code review, source control, documentation, observability, CI/CD, and infrastructure-as-code practices.
- Monitor, alert on, and provide business-hours support for healthy data pipelines and services.
- Collaborate with senior engineers and business stakeholders to deliver dependable data products.
Requirements
- Hands-on experience building and maintaining reliable, scalable data pipelines and data products, often alongside more senior engineers.
- Experience with Databricks, PySpark, Spark SQL, and Delta Lake or a similar lakehouse platform.
- Experience with AWS data and serverless services such as Lambda, Glue, S3, Kinesis, SNS, and SQS.
- Good Python and SQL skills, with understanding of data modelling and data quality fundamentals.
- Some experience with CI/CD and infrastructure as code.
- A degree in computer science or a related field, or equivalent knowledge and experience.
- Strong verbal and written communication skills and a curious, continuous-improvement mindset.
- Experience with dbt, Apache Airflow, Great Expectations, open table formats such as Delta, Apache Hudi, or Apache Iceberg, and Terraform is desirable.
Benefits
- Competitive salary and bonus, strong pension, private healthcare, flexible working, and learning and career growth support.
- Hybrid office/remote working arrangement.
- No travel is expected.
- Eligible for relocation assistance within the country.
About BP
BP is a global, publicly traded energy company that explores and produces oil and gas, refines and markets fuels and lubricants (including the Castrol brand), and trades energy products for consumers and industrial customers. It also operates convenience and retail fuel networks and invests in EV charging (bp pulse), bioenergy, and wind and solar projects. Founded in 1909 and headquartered in London, it is listed on the London and New York stock exchanges.