Senior Data Engineer
Sigma Software2 months ago
Remote, Poland or Warsaw, PolandSenior
Responsibilities
- Write and defend diagnostic SQL queries against large-scale production datasets.
- Build and maintain ingestion pipelines for bid, win, impression, and other event data in BigQuery.
- Harmonize independently designed datasets and maintain versioned field mappings.
- Develop point-in-time-correct feature tables, aggregation pipelines, conversion pipelines, and delayed-label handling.
- Own data-serving write paths, schema contracts, publishing flows, and freshness SLOs.
- Build experimentation infrastructure, including traffic splitting and reporting pipelines.
- Perform large-scale historical backfills and safe reprocessing after mapping changes.
- Implement advertiser data isolation and safe-aggregation controls.
- Develop automated data-quality validation frameworks.
- Collaborate with Customer engineers, produce operational documentation, and contribute to architecture and platform scalability improvements.
Requirements
- 5+ years of experience in Data Engineering.
- At least 2 years of experience with production ML or large-scale analytics pipelines.
- Expert-level SQL skills, including window functions and incremental processing patterns.
- Strong Python skills for production-grade pipeline development.
- Hands-on experience with Spark or PySpark.
- Experience designing ETL/ELT pipelines with Airflow, Cloud Composer, Dagster, or similar tools.
- Experience with cloud data warehouses at scale, preferably BigQuery.
- Strong understanding of data modeling and point-in-time correctness.
- Experience with very large-scale event or clickstream datasets and business-critical production pipelines.
- Upper-Intermediate English level or higher.
- Preferred experience with GCP services including Dataflow, Pub/Sub, GCS, and Beam.
- Preferred experience building streaming or near-real-time ingestion systems.
- Preferred knowledge of feature stores, train/serve skew, label leakage prevention, delayed or incomplete ML labels, dbt, customer-owned infrastructure delivery, GDPR/CCPA privacy engineering, experimentation infrastructure, hybrid cloud/on-prem environments, Linux, Terraform, Kubernetes, and warehouse cost and performance optimization.
- Strong analytical, problem-solving, communication, documentation, collaboration, and independent ownership skills.
Tech Stack
Apache AirflowApache BeamApache SparkdbtGoogle BigQueryGoogle Cloud PlatformKubernetesLinuxPythonSQLTerraform
Categories
Data Engineering
About Sigma Software
Sigma Software is a London-headquartered, privately held IT consultancy that designs and builds custom software for startups and enterprises. The company provides end-to-end services—engineering, web and mobile development, .NET/Java platforms, AR/VR, and UI/UX—delivered via outsourcing, managed teams, and consulting. Founded in 2002, it supports long-term product development and modernization projects across multiple industries for global clients.