Databricks Data Engineer and Scientist
Booz Allen Hamilton17 days ago
Base Salary
$78k - $176k/yr
Responsibilities
- Design, develop, deploy, and manage robust ETL processes and data pipelines with comprehensive logging and error handling.
- Build, debug, and optimize data pipelines using Databricks notebooks, clusters, job orchestration tools, Delta Lake, and Unity Catalog.
- Write Python and PySpark ETL scripts and custom transformations.
- Design data models using dimensional or relational approaches within Bronze, Silver, and Gold medallion architectures.
- Use AWS services to support data processing, storage, and database workloads.
- Optimize ETL and query performance through partitioning, bucketing, caching, and troubleshooting.
- Analyze and decompose legacy SQL, rebuild pipelines or models, and compare results for accuracy and performance.
- Support data security, compliance, governance, metadata management, data lineage, and data quality initiatives.
- Collaborate with cross-functional teams and explain complex technical concepts and architectural tradeoffs to non-technical audiences.
Requirements
- 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
- 3+ years of experience building, debugging, and optimizing pipelines with Databricks notebooks, clusters, and job orchestration tools.
- 3+ years of experience writing Python and PySpark for ETL scripts and custom transformations.
- Experience with Databricks, Delta Lake, Unity Catalog, data modeling, and medallion architecture.
- Experience with AWS Glue, Amazon S3, and Amazon RDS.
- Experience using AI-assisted coding tools such as Claude Code or Cursor for data engineering and modeling work.
- Knowledge of ETL performance optimization, legacy SQL analysis, pipeline validation, data governance, lineage, and compliance requirements such as GDPR and CCPA.
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
- Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology.
- Preferred experience with AWS Lambda, Amazon CloudWatch, AWS Glue Data Catalog, or Collibra.
Benefits
- Health, life, disability, financial, and retirement benefits.
- Paid leave, professional development, tuition assistance, work-life programs, and dependent care.
- Recognition awards for exceptional performance and demonstration of company values.
- Full-time and part-time employees regularly working at least 20 hours per week are eligible for Booz Allen benefit programs, with select offerings for employees below that threshold.
- The work model may be remote, hybrid, or onsite depending on the position listing, with possible work at Booz Allen or customer facilities and cameras generally expected during virtual meetings.
About Booz Allen Hamilton
Booz Allen Hamilton builds and integrates software, analytics, AI, cloud, and cybersecurity solutions while providing engineering and consulting services, primarily for U.S. defense, intelligence, and civil government clients, with select commercial partnerships. Its business is largely government contracting and professional services delivering mission systems and cyber defense. Founded in 1914 and headquartered in McLean, Virginia, Booz Allen is a public company traded on the NYSE (ticker: BAH).