Senior Engineer - Data Engineering
KSB SE & Co. KGaA2 hours ago
Pune, IndiaSenior
Responsibilities
- Design and develop scalable data pipelines and a growing data lake.
- Build and extend data processing workflows using Python, Apache Spark, and Databricks.
- Define technical standards, best practices, and reusable frameworks for data engineering.
- Ensure data quality, reliability, performance, and maintainability across data solutions.
- Support data modeling, data integration, and transformation for analytics and reporting.
- Drive automation, monitoring, and CI/CD improvements for operational excellence.
- Collaborate across engineering, data, analytics, and business teams as a technical interface.
- Contribute to architecture decisions and long-term data platform strategy.
- Deploy, monitor, test, and maintain data pipelines and services in production environments.
Requirements
- Bachelor’s degree in Software Engineering, Mathematics, Physics, or a related field.
- At least three years of project or coding experience in a company.
- Outstanding programming experience, preferably in Python, with the ability to write clean, testable, production-grade code.
- Strong SQL skills and familiarity with structured and semi-structured formats including JSON, Protobuf, and Delta format.
- Hands-on Apache Spark experience, ideally with Databricks, and understanding of medallion architecture.
- Knowledge of data lakehouse principles, data modeling, and data governance.
- Experience building and maintaining CI/CD pipelines, such as GitLab CI, plus familiarity with infrastructure as code and deployment.
- Experience with AWS or comparable cloud providers and familiarity with Databricks as a managed lakehouse platform.
- Experience with event-driven architectures or streaming platforms such as Kafka.
- Understanding of testing practices, Git, data versioning, package management, requirements and environments, change data capture, SQL partitioning, and relational data structures.
- Ability to work autonomously, take ownership end-to-end, and communicate clearly across engineering and data teams.
Tech Stack
Categories
Data Engineering