Senior Data Engineer

3 days ago

Remote, AmericasSenior

Base Salary

$80k - $120k/yr

Responsibilities

Design and build a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
Ingest data from APIs, webhooks, files, and web scraping while handling rate limits and failures.
Orchestrate scheduled workflows and historical backfills using Airflow, Dagster, Prefect, or similar tools.
Build messaging and task-queue workflows with retries, dead-letter queues, and controlled replay.
Store original payloads in cloud object storage and transform data with dbt into raw, standardized, and curated layers.
Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
Design and optimize PostgreSQL schemas and loading patterns, including enrichment and LLM-based structured extraction.

At least 5 years of experience building and operating production data platforms from architecture through deployment and ongoing operations.
Production experience with Google Cloud Platform, dbt, and workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal.
Strong Python skills, including asynchronous HTTP and concurrent processing with asyncio, aiohttp, or HTTPX.
Strong SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading.
Experience with event-driven systems, messaging or task queues, asynchronous processing, and failure recovery.
Experience with data contracts, raw and curated storage, retention policies, automated testing, and data quality monitoring.
Startup or small-team experience, web scraping, entity resolution, data enrichment, and LLM-based extraction are valuable.