Databricks Data Engineer and Scientist
Booz Allen Hamilton16 days ago
Base Salary
$78k - $176k/yr
Responsibilities
- Design, develop, deploy, and manage ETL processes and robust data pipelines with logging and error handling.
- Build, debug, and optimize Databricks notebooks, clusters, and job orchestration workflows using Delta Lake and Unity Catalog.
- Write Python and PySpark ETL scripts and custom transformations.
- Design dimensional or relational data models within Bronze, Silver, and Gold medallion architectures.
- Use AWS data services to support data processing and storage.
- Optimize partitioning, bucketing, caching, query performance, scalability, and pipeline reliability.
- Analyze complex legacy SQL, rebuild pipelines or models, and compare results for accuracy and performance.
- Support data governance, metadata, data lineage, privacy, security, and compliance activities.
- Collaborate with cross-functional teams and explain technical concepts and architectural tradeoffs to non-technical audiences.
Requirements
- 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
- 3+ years of experience building, debugging, and optimizing data pipelines with Databricks notebooks, clusters, and job orchestration tools, including Delta Lake and Unity Catalog.
- 3+ years of experience writing Python and PySpark for ETL scripts and custom transformations.
- Experience with AI-assisted coding tools such as Claude Code or Cursor for data engineering and data modeling.
- Experience with data modeling and Bronze, Silver, and Gold medallion architectures.
- Experience with AWS Glue, Amazon S3, and Amazon RDS.
- Knowledge of ETL performance optimization, complex legacy SQL analysis, pipeline rebuilding, and result validation.
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
- Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology.
- Experience with AWS Lambda, Amazon CloudWatch, AWS Glue Data Catalog, or Collibra is preferred.
- Knowledge of data governance, data lineage, GDPR, CCPA, data quality, privacy, and security is preferred.
Benefits
- Health, life, disability, financial, and retirement benefits are available to eligible employees.
- Paid leave, professional development, tuition assistance, work-life programs, dependent care, and recognition awards are offered.
- Full-time and part-time employees regularly working at least 20 hours per week are eligible for Booz Allen benefit programs, while others receive select offerings.
- The position is hybrid and may require work at a Booz Allen or customer facility according to leadership and role needs.
- Applicants selected will undergo a government investigation and may need to meet U.S. government client eligibility requirements.
About Booz Allen Hamilton
Booz Allen Hamilton builds and integrates software, analytics, AI, cloud, and cybersecurity solutions while providing engineering and consulting services, primarily for U.S. defense, intelligence, and civil government clients, with select commercial partnerships. Its business is largely government contracting and professional services delivering mission systems and cyber defense. Founded in 1914 and headquartered in McLean, Virginia, Booz Allen is a public company traded on the NYSE (ticker: BAH).