Senior Data Engineer (AWS & Databricks)
General Dynamics Information Technology23 hours ago
Remote, United StatesSenior
Base Salary
$111k - $150k/yr
Responsibilities
- Design, build, test, and maintain scalable, secure, and performant AWS and Databricks data platforms and services.
- Develop near-real-time and batch data ingestion, ETL/ELT, transformation, integration, data mart, and gold-layer pipelines.
- Implement data models, operational databases, metadata structures, governance workflows, security settings, and data classification controls.
- Integrate pipelines with cloud services, messaging, storage, APIs, and downstream reporting and analytics products.
- Implement data quality checks, validations, error handling, performance optimization, scalability improvements, and cost-efficiency measures.
- Support CI/CD-enabled deployments, automated testing, code reviews, design reviews, incident resolution, and root cause analysis.
- Execute data migration, archival, failover testing, disaster recovery, document storage, indexing, and secure API engineering activities using infrastructure as code.
- Produce technical documentation, runbooks, workflows, data models, testing artifacts, migration documentation, and audit-ready deliverables.
- Collaborate with architects, data governance and migration teams, DBAs, business analysts, QA personnel, scrum teams, and customer stakeholders.
- Mentor junior data engineers and support training, job aids, communications, and user adoption.
Requirements
- Bachelor’s degree in Data Science or a closely related discipline and 5+ years of specialized experience with AWS Cloud and Databricks.
- Deep hands-on experience with AWS data storage, compute, and serverless tools; Databricks; Apache Spark; and Delta Lake.
- Advanced proficiency in Python, PySpark, SQL, and data transformation techniques.
- Proficiency with ETL/ELT frameworks and hands-on experience building enterprise-scale data pipelines.
- Experience with streaming or near-real-time data ingestion, data governance, metadata, classification standards, and system security best practices.
- Familiarity with Git, containerization, CI/CD, Unity Catalog, Agile/Scrum delivery environments, and regulated or compliance-driven environments.
- Experience building and migrating software and IT services, researching advanced technologies, and applying industry tools, methods, best practices, and emerging technologies.
- Strong analytical, problem-solving, presentation, communication, consulting, stakeholder relationship, teamwork, and independent-work skills.
- Experience mentoring junior data engineers is required.
- Preferred qualifications include a bachelor’s degree with 7+ years of specialized experience or a master’s degree with 5+ years of specialized experience.
- Preferred certifications include Databricks Certified Data Professional, AWS Certified Data Analytics - Specialty, AWS Certified Solutions Architect - Professional, or AWS Certified Data Engineer.
- Must be a U.S. Person and obtain and maintain a Public Trust, including onsite fingerprinting and a background investigation.
Benefits
- Remote work from any location with less than 10% travel and a 40-hour scheduled work week.
- Medical plan options, including plans with Health Savings Accounts, dental plans, and vision plans.
- 401(k) plan with company matching and pre-tax and post-tax contribution options.
- Flexible work weeks where possible and paid vacation, sick, personal, holiday, parental, military, bereavement, and jury duty leave.
- Typically 15 days of paid leave per calendar year plus 10 paid holidays, prorated based on hire date.
- Up to 160 hours of paid family leave in a rolling 12-month period for eligible employees.
- Short- and long-term disability, life, accidental death and dismemberment, personal accident, critical illness, and business travel and accident insurance.
Tech Stack
Categories
Data Engineering
About General Dynamics Information Technology
General Dynamics Information Technology (GDIT) delivers enterprise IT, systems integration, cybersecurity, cloud, and software engineering services to U.S. federal agencies, including defense and intelligence organizations. It provides contract-based professional and managed services ranging from network operations and unified communications to training and simulation. GDIT is a business unit of General Dynamics (NYSE: GD) and is headquartered in Falls Church, Virginia.