Principal Databricks Software Developer
PowerPlan, Inc.2 months ago
Atlanta, GA, USAStaff+
Responsibilities
- Audit high-priority Databricks assets and deliver an evidence-backed assessment of platform risks.
- Design and drive adoption of Databricks compute policies, Spark configurations, notebook patterns, pipeline patterns, and Unity Catalog governance.
- Investigate complex Databricks failures, performance regressions, and cost anomalies to root cause.
- Create reusable defaults, templates, reference architectures, and automation instead of one-off fixes.
- Guide teams on Databricks design decisions and responsible adoption of AI-assisted engineering practices.
- Improve platform reliability, performance, governance, and cost-effectiveness.
Requirements
- 8+ years of experience in data engineering, platform engineering, or related software engineering roles.
- 3+ years of deep hands-on Databricks experience.
- Production experience with Spark internals and performance tuning, including shuffle behavior, query planning, adaptive query execution, memory pressure, skew, and partitioning.
- Strong knowledge of Databricks compute options, cluster policies, instance pools, autoscaling, Photon, and workload right-sizing.
- Experience reviewing and improving notebooks, jobs, and pipelines for runtime, cost, and reliability risks.
- Production experience with Delta Lake, Structured Streaming, Auto Loader, and data quality practices.
- Strong command of Unity Catalog governance, lineage, permission models, and secure handling of sensitive data.
- Experience designing resilient job orchestration with dependency management, retries, timeouts, and alerting.
- Strong communication skills and the ability to translate technical risk into actionable stakeholder language.
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent professional experience.
Benefits
- Hybrid work arrangement combining onsite work at the corporate office with work from home.
- Onsite attendance may be required for scheduled office days, team meetings, client meetings, or special events.
Tech Stack
Apache SparkDatabricks
Categories
Data EngineeringDevOps
About PowerPlan, Inc.
PowerPlan builds enterprise software that helps asset‑intensive organizations—especially utilities, energy, and other infrastructure operators—manage asset accounting, tax, and regulatory finance. It sells a suite now delivered on a modernizing SaaS platform, paired with professional services for implementations, integrations, and cloud migrations. Founded in 1994 and headquartered in Atlanta, it operates as an independent subsidiary of Roper Technologies and is used across sectors including energy, telecommunications, and rail.