22 days ago
Chicago, IL, USA +2 moreStaff+
Base Salary
$118k - $206k/yr
Responsibilities
- Lead the design, implementation, and continuous evolution of the enterprise Databricks platform and supporting architecture.
- Develop reusable platform services, reference architectures, engineering standards, and implementation patterns for analytics, data products, streaming, and AI workloads.
- Build infrastructure automation, CI/CD pipelines, and reusable frameworks to improve platform scalability, reliability, and operational efficiency.
- Create self-service capabilities, reusable templates, and engineering accelerators for platform onboarding and deployment.
- Optimize platform performance, scalability, security, governance, and cost efficiency across cloud environments.
- Implement Databricks patterns for MLflow, Model Registry, Feature Store, Vector Search, and other machine learning and AI capabilities.
- Partner with Enterprise Architecture, Data Engineering, Product Management, Cloud Engineering, Data Science, and client stakeholders.
- Evaluate emerging Databricks features and cloud technologies, lead proofs of concept, and recommend platform enhancements.
- Provide technical leadership, mentoring, and engineering guidance while promoting Platform-as-a-Product principles and continuous improvement.
Requirements
- At least 7 years of relevant experience.
- At least 3 years of hands-on experience designing, implementing, and supporting Databricks platform solutions.
- Strong experience with Azure cloud services, cloud-native architectures, and enterprise platform engineering.
- Experience with Terraform, infrastructure automation, Infrastructure-as-Code, and CI/CD pipelines.
- Strong knowledge of Delta Lake, Unity Catalog, Apache Spark, and lakehouse architectures.
- Experience developing reusable platform services, automation frameworks, self-service capabilities, and engineering accelerators.
- Preferred qualifications include a relevant degree or advanced degree in Computer Science, Engineering, Information Systems, or a related field.
- Preferred qualifications include machine learning and AI platform experience, Databricks certification, streaming and event-driven architecture experience, enterprise-scale data platform experience, or healthcare industry experience.
Benefits
- Comprehensive benefits plan.
- The position is incentive eligible.
- The salary range is geographically adjusted, and the role may involve working across cloud environments; specific remote or office arrangements are not stated.
Tech Stack
Categories
