4 months ago
Houston, TX, USAMid Level
Responsibilities
- Design, build, and maintain time-critical ETL/ELT data integration pipelines across internal and external data sources.
- Develop and maintain data models for varied datasets flowing through the integration layer.
- Implement and manage cloud-based data integration solutions using AWS services including S3, Glue, Lambda, and Redshift.
- Monitor, troubleshoot, and optimize pipelines for reliability, performance, and data quality.
- Automate data workflows for the North American Gas and Power trading desk to reduce manual intervention and operational risk.
- Understand the desk’s data landscape and commercial requirements to deliver effective integration solutions.
- Collaborate with traders, commercial stakeholders, and the Data Science and Engineering team.
Requirements
- At least 3 years of experience with Python and SQL, focused on data integration or pipeline development.
- Experience with relational databases and data modeling.
- Hands-on experience with cloud data services such as AWS, Azure, or GCP.
- Experience with containerization such as Docker or Kubernetes and infrastructure-as-code such as Terraform or CloudFormation.
- Experience monitoring, observing, and debugging data pipelines using logging, alerting, and observability tools such as CloudWatch or Grafana.
- Proficiency with AI assistant tools such as GitHub Copilot or Claude to support development workflows.
- Bachelor’s degree in Computer Science, Data Science, or a related STEM field.
- Preferred qualifications include AWS certifications, NoSQL database experience, familiarity with streaming or event-driven integration patterns, and exposure to Gas or Power trading environments.
- Strong ownership of data quality and pipeline reliability, clear communication skills, and the ability to work under pressure in a time-sensitive trading environment.
Tech Stack
Categories
Data Engineering
