Data Engineer - AWS/Databricks - Mid Level
Acuity, Inc.2 years ago
Reston, VA, USAMid Level
Base Salary
$120k - $140k/yr
Responsibilities
- Build and maintain scalable PySpark data pipelines in Databricks for structured and semi-structured data ingestion, transformation, and enrichment.
- Design optimized Delta Lake tables and develop ETL and ELT workflows for centralized data warehouse architectures.
- Use Spark SQL and DataFrame APIs to implement business rules, dimensional joins, aggregations, and warehouse modeling logic.
- Implement cloud-native data solutions on AWS using S3, Glue, RDS, and IAM.
- Optimize pipeline performance and cost through partitioning, caching, broadcast joins, and adaptive query tuning.
- Deploy and version data engineering assets using Git, GitLab, Jenkins, and CI/CD workflows.
- Monitor Databricks pipelines and AWS infrastructure with native Databricks tools and AWS CloudWatch.
- Conduct legacy-system discovery, source mapping, transformation design, and end-to-end data-flow planning.
- Implement data governance practices including metadata tagging, data quality validation, audit logging, and lineage tracking.
- Support data access requests, reusable data assets, shared notebooks, and operational reporting and analytics needs.
Requirements
- At least 4 years of experience in data engineering and Agile analytics.
- At least 4 years of experience creating software for retrieving, parsing, and processing structured and unstructured data.
- At least 2 years of experience building scalable ETL and ELT workflows for reporting and analytics.
- At least 2 years of experience building enterprise cloud data engineering solutions, preferably using AWS and Databricks cloud-native technologies.
- Experience with data quality, validation frameworks, and storage optimization strategies.
- Bachelor’s degree or equivalent BA/BS education.
- Must be a US citizen with the ability to obtain and maintain US Suitability.
Tech Stack
Categories
Data Engineering
About Acuity, Inc.
Acuity, Inc. is a federal IT consulting firm that designs and implements technology solutions for U.S. government agencies, with practices in IT modernization, cloud, data engineering and analytics, cybersecurity, and hyperautomation. It delivers program management and systems integration services, including AI/ML and secure cloud deployments compliant with NIST and FedRAMP. The company focuses on the U.S. public sector and operates in highly regulated environments supporting defense, intelligence, and civilian missions.