17 days ago
Responsibilities
- Build and scale the data platform based on Databricks for product features and internal reporting.
- Design and develop reliable and scalable data ingestion and transformation pipelines.
- Model raw data into clean and reliable datasets for product functionalities and internal analytics.
- Create abstractions and platform tools to enhance developer experience for data producers and consumers.
- Improve observability, testing, and monitoring to ensure reliability and performance.
- Manage and contribute to data governance, security, compliance, and access controls.
- Contribute to platform standards and architecture as it evolves.
- Collaborate with development, machine learning, product, and analytics teams to support their data use cases.
Requirements
- Knowledge of regulatory and compliance frameworks such as GDPR, CCPA, SOC2, and FedRAMP.
- Experience in backend software development, including application data storage and event-driven architecture.
- Experience in creating and operating vector databases and RAG.