2 days ago
Pune, IndiaStaff+
Responsibilities
- Design end-to-end data and AI solutions, translating high-level architecture into buildable HLD and LLD specifications.
- Design, build, and maintain scalable data pipelines, data products, and enterprise integration layers.
- Establish engineering standards for software design, code quality, testing, CI/CD, observability, and documentation.
- Integrate LLMs, retrieval-augmented generation, AI agents, machine learning capabilities, and unstructured-data processing into production applications.
- Evaluate build, configure, SaaS, platform, and low-code/no-code options with attention to reuse, cost, maintainability, and scalability.
- Work with architecture, cyber, data governance, and platform teams on security, resilience, compliance, performance, and operational readiness.
- Provide technical direction across internal teams, delivery partners, and third-party suppliers without direct line management.
- Design for reliability, supportability, observability, SLA compliance, and cost effectiveness throughout the solution lifecycle.
- Prototype practical technology solutions and validate ideas with users before larger-scale implementation.
- Mentor engineers and contribute to data engineering standards, reusable patterns, and engineering communities.
Requirements
- Bachelor’s degree in Computer Science or a related field.
- 8+ years of hands-on experience designing, building, productionising, and maintaining reliable, scalable data infrastructure and data products in complex environments.
- Experience producing HLD and LLD artifacts, architecture diagrams, data-flow designs, component specifications, API contracts, data models, and operational runbooks.
- Ability to make architectural decisions and decompose them into implementable engineering specifications.
- Strong software engineering practices, including design reviews, unit testing, CI/CD pipeline development, monitoring, alerting, code review, and documentation.
- Development experience in an object-oriented programming language; Python is preferred, with Scala, Java, or C# also applicable.
- Expert SQL knowledge and experience with large-scale distributed data systems.
- Experience integrating AI and machine learning capabilities into production systems, including LLM APIs, RAG, vector search, embeddings, or AI agents.
- Deep knowledge and hands-on experience across the full data lifecycle.
- Strong stakeholder management and technical-influence skills across internal teams and external partners.
- Experience with AI/ML frameworks such as LangChain, LlamaIndex, Hugging Face, or Weaviate is desired.
- Familiarity with agentic frameworks such as CrewAI or AutoGen is desired.
- Experience with Spark, Hadoop, or Hive is desired.
- Familiarity with AWS, Azure, or GCP cloud data platforms and infrastructure-as-code practices is desired.
- Experience with Alation or Collibra and enterprise data management standards is desired.
Benefits
- Hybrid office/remote working arrangement.
- Up to 10% travel expected.
- Relocation assistance within the country.
- Flexible working options and collaboration spaces in a modern office environment.
- Access to diversity, equity, and inclusion business resource groups, mentoring, and development opportunities.
- Equal opportunity employment and reasonable accommodation support.
Tech Stack
Categories
Data Engineering
About BP
BP is a global, publicly traded energy company that explores and produces oil and gas, refines and markets fuels and lubricants (including the Castrol brand), and trades energy products for consumers and industrial customers. It also operates convenience and retail fuel networks and invests in EV charging (bp pulse), bioenergy, and wind and solar projects. Founded in 1909 and headquartered in London, it is listed on the London and New York stock exchanges.