about 3 hours ago
Responsibilities
- Develop and maintain high-performance, distributed big data platforms.
- Build innovative tools from scratch while improving existing infrastructure.
- Collaborate with data engineers and ML scientists to roll out tools.
- Design and build data contracts, validation frameworks, and optimization tooling.
- Develop stream processing applications to enhance data insights.
Requirements
- 6+ years of experience in object-oriented programming with Python or Java.
- Proven experience in developing and maintaining distributed compute/data systems.
- Experience with real-time and batch pipelines using Kafka and Spark Streaming.
- Strong communication skills to enable collaboration across teams.
- Familiarity with metadata collection, data lineage, and data quality tools.