Cloud Data Engineer (50576)
GAP Solutions, Inc.8 months ago
Atlanta, GA, USAMid Level
Responsibilities
- Collaborate with scientists, data engineers, developers, and data consumers in a multidisciplinary Agile environment.
- Monitor and optimize data pipelines for performance, scalability, and cost-effectiveness.
- Support the assessment, design, development, and maintenance of scalable client data platforms.
- Develop and maintain ETL pipelines using Azure technologies, including Azure Batch and Azure Data Factory.
- Support analytical exploration, data examination, and large-scale data processing.
Requirements
- Bachelor’s degree required.
- At least 3 years of experience with ETL operations focused on Azure technologies.
- At least 2 years of experience with source control and collaboration software, including Git or Atlassian tools.
- Knowledge of Azure Batch and its application to large datasets.
- Experience with SQL and relational databases such as Azure SQL Database and SQL Server.
- Experience with Python or R and data-manipulation libraries such as Pandas, NumPy, Polars, or Tidyverse.
- Proficiency with Azure Data Factory and its components.
- Preferred experience developing pipelines with Azure Batch and Azure Data Factory.
- Preferred familiarity with Apache Airflow or similar workflow orchestration tools.
- Preferred experience with Azure Synapse Analytics, Azure Databricks, or Azure Blob Storage.
- Familiarity with cloud security best practices and data governance.
- Ability to learn technical concepts quickly and communicate with multiple functional groups.
- Strong problem-solving skills and ability to work independently and collaboratively.
Categories
Data Engineering
About GAP Solutions, Inc.
GAP Solutions, Inc. provides mission support, consulting, and staffing services to U.S. federal civilian and defense agencies, with offerings spanning emergency management, security and information management, process management, human capital, and scientific research and policy analysis. The privately held company, founded in 1999 and headquartered in Herndon, VA, delivers work under government contracts; its past performance includes agencies such as HHS, DoD, State, and DHS. GAP holds ISO 9001:2015 quality certification.