
Data Engineer
Bristlecone12 hours ago
Bengaluru, IndiaSenior
Responsibilities
- Develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications.
- Build scalable backend services, data platforms, data pipelines, and data transformation processes.
- Use Spark, SQL, Hive, Python, Scala, and AWS services to develop cloud-native data and backend solutions.
- Design and enhance Apache Airflow DAGs, including scheduling, dependency management, monitoring, and failure handling.
- Define technical scope, objectives, and implementation approaches through requirements gathering, technical research, and process definition.
- Participate in architecture reviews, code reviews, performance tuning, test planning, validation, and operational readiness activities.
- Troubleshoot production issues, improve system reliability, and automate data and backend workflows.
- Collaborate with product owners, data engineers, backend engineers, QA, DevOps, and other cross-functional teams.
Requirements
- At least 5 years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems.
- At least 5 years of experience developing and operating Big Data platforms and cloud-based infrastructure services, preferably with AWS EMR and the Hadoop ecosystem.
- Hands-on experience with AWS, Spark, Python or Scala, Apache Airflow, SQL, Hive, and related data processing technologies.
- Proficiency in Python or Scala with the ability to build production-grade data pipelines and backend services.
- Strong experience designing and operating complex Apache Airflow workflows.
- Strong understanding of data pipelines, data transformation, batch processing, real-time processing, data quality, and performance optimization.
- Experience with version control and CI/CD tools including Git and Jenkins.
- Experience with API design, microservices, event-driven architecture, serverless backend development, infrastructure as code, automated testing, and production deployment practices.
- Experience with LLMs, Generative AI, Agentic AI, or AI-assisted engineering workflows.
- Experience with data governance, metadata management, lineage, data observability, or operational controls for enterprise data platforms.
- Strong analytical, problem-solving, communication, and presentation skills, including the ability to explain technical concepts to technical and non-technical audiences.
- Ability to work effectively in cross-functional Agile/Scrum teams.
Tech Stack
Categories
BackendData Engineering
About Bristlecone
Bristlecone provides supply chain consulting and systems integration, delivering planning, sourcing, and fulfillment solutions for Global 2000 enterprises. Its teams implement platforms such as SAP Integrated Business Planning and Kinaxis, alongside analytics, data, and integration services. Founded in 1998 and headquartered in San Jose, it operates worldwide and is part of the Mahindra Group. Its business model centers on advisory, implementation, and managed services for life sciences, retail, consumer goods, manufacturing, and high tech.