Lead Data Engineer
Redhorse Corporation20 days ago
Remote, United StatesStaff+
Base Salary
$170k - $190k/yr
Responsibilities
- Provide day-to-day task prioritization, technical direction, code reviews, mentoring, and design guidance to engineers.
- Elicit and translate government mission requirements into technical specifications, interface designs, and scalable data models.
- Architect, develop, optimize, maintain, and refactor high-throughput ETL/ELT workflows in Databricks.
- Develop and optimize modular Python/PySpark code and complex SQL queries for performance, stability, security, and data integrity.
- Diagnose pipeline bottlenecks, debug complex data workflows, and implement automated reliability improvements.
- Collaborate with government stakeholders, product owners, data scientists, data analysts, and technical peers.
- Create and maintain pipeline architecture documentation and configuration guidance.
- Build functional, user-centric reports and dashboards using modern data visualization tools.
Requirements
- U.S. citizenship and a current, active U.S. Government Secret clearance are required.
- At least 8 years of professional experience in data engineering and building enterprise-scale data platforms and pipelines.
- Demonstrated experience mentoring engineers, performing code reviews, and providing technical task oversight.
- Hands-on experience designing and optimizing Databricks data transformation workflows.
- Advanced modular, object-oriented Python/PySpark skills and extensive experience writing, tuning, and optimizing complex SQL queries.
- Proven experience building functional reports and dashboards with modern data visualization tools.
- Preferred experience includes Apache Spark, distributed computing frameworks, DoD or intelligence community data environments, compliance standards, data governance, metadata management, data cataloging, GitLab, and Jira.
- A bachelor's or master's degree in Computer Science, Information Systems, Engineering, or a related technical discipline is preferred.
Benefits
- The posting references comprehensive benefits programs and possible performance-based or other incentive compensation for eligible employees.
- The role supports U.S. national defense missions and requires an active Secret clearance; no work arrangement or other practical logistics are stated.
Tech Stack
Categories
Data Engineering
About Redhorse Corporation
Redhorse Corporation provides data analytics, AI, and digital transformation consulting to U.S. government agencies, especially in national security, infrastructure, and environmental programs. It delivers technology-agnostic solutions, systems integration, and data engineering to help the Department of Defense and civilian agencies improve decision-making and operations under professional services contracts. Founded in 2008 and headquartered in Arlington, Virginia, the company is privately held.