Databricks Data Engineer, Lead
Booz Allen Hamilton28 days ago
Base Salary
$78k - $176k/yr
Responsibilities
- Design, build, deploy, and maintain end-to-end ETL processes and robust data pipelines in Databricks and AWS.
- Create, optimize, and troubleshoot data flows using Databricks notebooks, clusters, job orchestration, Delta Lake, Databricks Delta, and Unity Catalog.
- Write Python and PySpark ETL scripts and custom data transformations.
- Design schemas and data models using dimensional and relational approaches within Bronze, Silver, and Gold medallion architectures.
- Optimize ETL and query performance through partitioning, bucketing, caching, and troubleshooting.
- Design and maintain scalable data platforms while addressing data security, governance, compliance, logging, and error handling.
- Mentor analysts, data engineers, developers, and data consumers and provide technical leadership on architectural decisions.
- Collaborate with cross-functional teams and explain complex technical concepts and design tradeoffs to non-technical audiences.
Requirements
- At least 8 years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
- At least 3 years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools.
- At least 3 years of experience writing Python and PySpark code for ETL scripts and custom transformations.
- Experience with data modeling, medallion architectures, dimensional and relational models, and efficient query-oriented data structures.
- Experience with AWS services including AWS Glue, Amazon S3, and Amazon RDS.
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
- Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology.
- Preferred experience with AWS Lambda, CloudWatch, AWS Glue Data Catalog, or Collibra.
- Preferred knowledge of data governance, data lineage, GDPR, CCPA, data quality, privacy, and security.
Benefits
- Hybrid work arrangement in Bethesda, Maryland, or Charleston, South Carolina, with facility attendance aligned to leadership expectations and role needs.
- Health, life, disability, financial, and retirement benefits.
- Paid leave, professional development, tuition assistance, work-life programs, and dependent care benefits.
- Recognition awards and access to Booz Allen’s broader employee benefits programs for eligible full-time and part-time employees working at least 20 hours per week.
About Booz Allen Hamilton
Booz Allen Hamilton builds and integrates software, analytics, AI, cloud, and cybersecurity solutions while providing engineering and consulting services, primarily for U.S. defense, intelligence, and civil government clients, with select commercial partnerships. Its business is largely government contracting and professional services delivering mission systems and cyber defense. Founded in 1914 and headquartered in McLean, Virginia, Booz Allen is a public company traded on the NYSE (ticker: BAH).