Senior Engineer - Data Ingestion
CTC Lateral2 months ago
Base Salary
$200k - $250k/yr
Responsibilities
- Own the quality, reliability, and overall health of large-scale source data pipelines.
- Partner with traders and quantitative researchers to understand stakeholder needs and data usage.
- Design reliable and efficient approaches for sourcing and delivering required data.
- Optimize pipelines to improve service-level performance and reduce compute and storage usage.
- Migrate pipelines to modern data platforms and hardware and improve future migration processes.
- Collaborate with the broader data organization to improve platform functionality.
- Detect source data quality issues and build tools with data owners to surface and resolve them before downstream users are affected.
Requirements
- Hands-on experience building and operating large-scale data pipelines, ideally at hundreds-of-terabytes to petabytes scale.
- Strong software architecture and large-scale system design skills.
- Strong proficiency in SQL and object-oriented languages such as Python.
- Hands-on experience with at least one major cloud provider, including AWS, Azure, or GCP, and/or HPC workloads.
- Strong interpersonal skills and ability to work in a team environment.
- Experience with tick capture, market data, or trading industry data is preferred.
- C++ experience is preferred.
- Experience with data observability and data quality tooling is preferred.
- Experience with open table formats such as Iceberg or Delta and query engines such as Trino, Spark, or DuckDB is preferred.
Benefits
- Annual discretionary bonus eligibility.
- Hybrid workplace model for most teams, subject to change based on business need.
- Generous time off, insurance coverage, paid parental leave, free breakfast and lunch, healthy snacks, wellness reimbursement, and other employee benefits.
Categories
Data Engineering
About CTC Lateral
Chicago Trading Company (CTC) is a proprietary trading firm specializing in options market making and other derivatives across global exchanges. It trades its own capital, combining quantitative research, systematic strategies, and in-house trading technology to provide liquidity and price discovery. Founded in 1995 and headquartered in Chicago, the privately held firm operates in multiple asset classes and supports research and trading with extensive market and reference data platforms.