Data Engineer
Black Canyon Consulting2 months ago
Bethesda, MD, USAMid Level / Senior
H1B sponsor
Responsibilities
- Design, implement, and maintain scalable data pipelines for analytical and AI/ML workloads.
- Build, monitor, troubleshoot, and optimize ETL and ELT pipelines.
- Develop workflows for ingesting, transforming, processing, and delivering data.
- Configure and integrate databases, data lakes, data warehouses, feature stores, and other data platforms.
- Develop and maintain data transformation and feature engineering processes.
- Develop APIs and service layers exposing data or model endpoints to downstream systems.
- Ensure data reliability, accessibility, quality, and operational readiness.
- Monitor pipeline performance and resolve data processing or integration issues.
- Implement data validation and data quality processes.
- Support deployment and operation of data infrastructure throughout the system lifecycle.
- Improve pipeline scalability, performance, maintainability, and efficiency.
- Document pipelines, integrations, transformations, and technical processes.
Requirements
- Experience designing, developing, and maintaining data pipelines.
- Experience building and optimizing ETL and/or ELT workflows.
- Experience with relational and/or non-relational databases and data platforms.
- Experience developing data transformation workflows and supporting analytical, AI, or machine learning workloads.
- Experience developing or integrating APIs and service layers.
- Understanding of data reliability, accessibility, quality, operational readiness, data modeling, and data architecture.
- Experience monitoring, troubleshooting, and optimizing data pipelines.
- Strong programming and problem-solving skills.
- Ability to collaborate with software engineers, data scientists, analysts, and other technical stakeholders.
- Junior level requires 3+ years of relevant experience; mid-level requires 5+ years; senior requires 8+ years.
- Desired qualifications include experience with machine learning or artificial intelligence environments, feature stores, distributed systems, cloud-based data platforms, automated monitoring and alerting, data lakehouse or warehouse architectures, and data validation, testing, and quality controls.
Benefits
- Medical, dental, and vision coverage.
- 401k plan with employer contribution.
- Paid holidays and vacation.
- Tuition reimbursement.
Categories
Data Engineering
About Black Canyon Consulting
Black Canyon Consulting is a privately held, woman-owned IT and program management services firm that delivers contract support to U.S. federal research and health agencies. The company provides systems engineering, custom software, bioinformatics, and project management through government contracts and IDIQ vehicles; it was founded in 2017 and is headquartered in Fairfax, Virginia. Recent roles support the National Center for Biotechnology Information at the NIH/NLM.