Databricks Data Engineer, Senior
Booz Allen Hamilton3 days ago
Base Salary
$78k - $176k/yr
Responsibilities
- Design, build, deploy, maintain, and optimize end-to-end data pipelines and ETL processes.
- Develop Databricks notebooks, clusters, and job orchestration workflows using Delta Lake, Databricks Delta, and Unity Catalog.
- Write Python and PySpark code for ETL scripts and custom transformations.
- Design schemas and data models, including dimensional and relational models within a Bronze, Silver, and Gold medallion architecture.
- Use AWS data services to build and support scalable data processing platforms.
- Apply partitioning, bucketing, caching, logging, and error handling to improve performance and troubleshoot pipeline issues.
- Support data governance, metadata management, lineage, privacy, security, and compliance requirements.
- Collaborate with cross-functional teams and mentor analysts, data engineers, developers, and data consumers.
- Provide technical leadership and explain complex technical concepts and architectural tradeoffs to non-technical audiences.
Requirements
- At least 8 years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
- At least 3 years of experience building, debugging, and optimizing data pipelines with Databricks notebooks, clusters, and job orchestration tools.
- At least 3 years of experience writing Python and PySpark for ETL scripts and custom transformations.
- Experience with data modeling, schemas, dimensional and relational models, and medallion architectures.
- Experience with AWS Glue, Amazon S3, and Amazon RDS.
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
- Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, Information Technology, or a related listed field.
- Preferred experience with AWS Lambda, CloudWatch, AWS Glue Data Catalog, or Collibra.
- Preferred knowledge of data governance, data lineage, GDPR, CCPA, data quality, privacy, and security.
- Ability to communicate complex technical concepts and architectural tradeoffs to non-technical audiences.
Benefits
- Health, life, disability, financial, and retirement benefits.
- Paid leave, professional development, tuition assistance, work-life programs, and dependent care.
- Recognition awards for exceptional performance and demonstration of company values.
- Eligible full-time and part-time employees working at least 20 hours per week can participate in benefit programs, with select offerings for employees below that threshold.
- Work model may be remote, hybrid, or onsite depending on the position listing and customer or leadership requirements; remote employees may occasionally work at a Booz Allen or customer facility.
- Applicants selected for the role will undergo a government investigation and may need to meet U.S. government client eligibility requirements.
About Booz Allen Hamilton
Booz Allen Hamilton builds and integrates software, analytics, AI, cloud, and cybersecurity solutions while providing engineering and consulting services, primarily for U.S. defense, intelligence, and civil government clients, with select commercial partnerships. Its business is largely government contracting and professional services delivering mission systems and cyber defense. Founded in 1914 and headquartered in McLean, Virginia, Booz Allen is a public company traded on the NYSE (ticker: BAH).