3 days ago
USA (city not specified) +3 moreMid Level
Base Salary
$55k - $999k/yr
Responsibilities
- Build scalable batch and streaming data pipelines optimized for latency, cost, and correctness.
- Process near-real-time events with deduplication, windowing, and late or out-of-order event handling.
- Own ingestion from databases using CDC, files through SFTP and GCS, APIs, and onchain sources.
- Build incremental dbt models supporting late data, duplicates, CDC deletes, backfills, and reconciliation between streaming and batch outputs.
- Orchestrate pipelines in Dagster using sensors, schedules, and alerts.
- Implement freshness, volume, reconciliation, monitoring, and data quality checks and resolve root causes.
- Collaborate with backend teams on event and table contracts.
- Partner with Analytics Engineers, Data Scientists, and Risk teams on data needs and ML features.
Requirements
- At least 3 years of experience building production data solutions and owning batch and streaming pipelines end to end.
- Experience designing data solutions at scale.
- Strong SQL and Python or Rust skills.
- Experience with a cloud data warehouse such as BigQuery, ClickHouse, or Snowflake and understanding of their trade-offs.
- Streaming experience with Pub/Sub, Kafka, Dataflow/Beam, or Flink.
- Batch experience with dbt and an orchestrator such as Dagster or Airflow.
- Experience with production data quality testing, monitoring, and reconciliation.
- Ability to debug across source databases, pipelines, warehouses, and reports.
- Ability to use AI tools to work faster while checking their output.
- Preferred experience building offline and real-time ML feature pipelines, working with payments, banking, or blockchain data, using EVM or Solana events and indexers, using CDC and ingestion tools such as Datastream, Debezium, Meltano, Fivetran, or Airbyte, and working with GCP.
Benefits
- Flexible workplace with work-from-home, office, or hybrid options.
- Unlimited vacation with a required minimum of 10 days off annually.
- Home-office setup stipend for new employees.
- For US employees, 95% coverage of employee health, dental, and vision plans and 90% coverage for dependents.
- Company-subsidized life insurance.
- 401(k) with a 4% company match.
- Equity option plan.
- Monthly health and wellness stipend.
- Lunch and dinner covered for office-based employees through DoorDash credit.
- Domestic and international team and company offsites.
Tech Stack
AirbyteApache AirflowApache BeamApache FlinkApache KafkaClickHousedbtGoogle BigQueryGoogle Cloud PlatformPythonRustSnowflakeSQL
Categories
Data Engineering
About Rain
Rain builds a card issuance and payments infrastructure platform for enterprises, fintechs, and platforms to use tokenized money in real-world transactions. Its services include compliant program management, digital-asset authorization and settlement, and rails that power stablecoin-backed cards, cross-border payouts, B2B purchases, and remittances. Founded in 2021, the privately held company partners with financial institutions and neobanks worldwide and is backed by investors such as ICONIQ, Lightspeed, Khosla, Galaxy, Sapphire, and Norwest.