2 months ago
Houston, TX, USAMid Level
Responsibilities
- Build, test, operate, monitor, and support production data pipelines in Python using Prefect.
- Build and maintain clean, tested, documented, and cost-aware Snowflake warehouse models with dbt.
- Migrate legacy pipelines and Oracle-based components to current frameworks and standards.
- Work across Kafka, Redis, InfluxDB, Oracle, Snowflake, and AWS data services according to established platform patterns.
- Acquire and reliably land data from vendor APIs, files, feeds, and web sources.
- Implement data quality, freshness, reconciliation, monitoring, alerting, and failure-recovery capabilities.
- Contribute to making platform data AI-ready and coordinate with catalog and data steward teams on documentation, classification, and discoverability.
- Engage directly with analysts and desk users to clarify needs and validate delivered solutions.
Requirements
- At least 3 years of hands-on experience building and operating production data pipelines.
- Strong Python skills and fluency with pandas and the broader data-handling ecosystem.
- Deep SQL skills, including data modeling, querying, tuning, and understanding query cost.
- Experience with Snowflake or a comparable cloud warehouse, including dimensional modeling, performance tuning, and cost-aware design.
- Experience with dbt is a strong plus.
- Breadth across streaming, caching, timeseries, relational, and warehouse technologies; experience with Kafka, Redis, InfluxDB, Oracle, and Snowflake or comparable technologies is relevant.
- Experience with pipeline orchestration; Prefect is preferred, while Airflow, Dagster, or similar tools are considered.
- Sound engineering fundamentals including object-oriented design, design patterns, testing, code review, and version control.
- Working knowledge of AWS data services and reliable system composition.
- A build-to-operate mindset covering monitoring, alerting, and failure recovery.
- Clear communication and the ability to explain technical trade-offs and turn vague requests into defined requirements.
- Exposure to Terraform, Docker, or API development with FastAPI or Flask is welcome.
- Experience in financial services, commodities, or energy trading data is a plus.
Benefits
- Medical, dental, and vision coverage.
- Paid vacation.
- 401(k) with company contributions.
- Life insurance.
- Short-term and long-term disability coverage.
- The posting states that all information will be kept confidential according to EEO guidelines.
Tech Stack
Categories
Data Engineering
About Vitol
Vitol trades, transports, and distributes energy and commodities—from crude and refined products to gas, power, renewables and carbon credits—for refineries, utilities, airlines, retailers and other traders. It also operates and invests in supporting infrastructure, including shipping, terminals and storage, and power assets. Founded in 1966 and privately held, Vitol is headquartered in Rotterdam and runs a global business from 40+ offices.