4 days ago
Remote, PortugalStaff+
Responsibilities
- Set architecture and technical direction for the Query Planner and Reflections team across open-source, enterprise, and cloud editions.
- Own the design, implementation, and delivery of complex cross-team initiatives involving Query Execution, Catalog, and Datalake teams.
- Work deeply with Apache Calcite and Apache Iceberg and drive upstream contributions when beneficial.
- Diagnose difficult query-planning and reflection-matching correctness and performance issues using query profiles, telemetry, and production data.
- Design architectures that operate correctly across deployment models and public clouds.
- Conduct design and code reviews, establish technical standards, and help other engineers grow.
- Partner with Product on multi-quarter roadmap decisions and with Support and field teams on high-impact customer issues.
- Use AI-assisted development tools for code comprehension, debugging, test generation, documentation, and incident analysis while validating generated code through review, testing, and benchmarking.
Requirements
- Bachelor’s or master’s degree in computer science or a related technical field, or equivalent experience.
- 8–10+ years of experience developing production-level software.
- Deep understanding of query planning and optimization, including cost-based optimizers and rule-based rewriting.
- Strong database fundamentals including SQL, performance, and schema design.
- Track record of driving system-level architecture decisions and leading complex cross-team initiatives.
- Experience setting technical direction for a team or major component and navigating ambiguous cross-organizational problems independently.
- Fluency in Java or C++ is a plus.
- Experience with Apache Calcite or a similar query-planning framework is preferred.
- Understanding of distributed file systems such as S3, ADLS, or HDFS is preferred, and AWS, Azure, or GCP experience is a plus.
- Bonus experience includes distributed query engines, query processing or optimization, materialized view matching, distributed systems, concurrency control, code generation, Apache Arrow, caching, disk spilling, and contributions to Apache Calcite, Apache Iceberg, or similar open-source planner projects.
About Dremio
Dremio builds a unified data lakehouse platform for self-service analytics and AI, enabling SQL-based access and federated queries across cloud, hybrid, and on‑prem data, powered by Apache Iceberg and Apache Arrow. It sells enterprise software and cloud services to data engineering, BI, and data science teams; customers include Maersk and S&P Global. Founded in 2015 and headquartered in Austin, Dremio is now part of SAP.