3 hours ago
New York, NY, USAMid Level
Base Salary
$200k - $200k/yr
Responsibilities
- Design and ship backend systems for multi-stage, high-volume data processing.
- Define versioned contracts for manifests, artifacts, lineage, and state transitions.
- Make retries, checkpoints, partial failures, replay, backfills, migrations, and rollbacks safe and understandable.
- Build independent reconciliation and verification to detect incorrect output beyond basic job success.
- Convert recurring failures into fixtures, regression coverage, release gates, and durable recovery paths.
- Expose trustworthy pipeline state and safe controls for investigating and releasing data.
- Diagnose production behavior across code, queues, stores, artifacts, data formats, and deployed versions.
- Improve correctness, throughput, and operating leverage while maintaining privacy, security, and release confidence.
- Use AI in development and bounded verification systems with explicit evaluation and independent checks.
- Collaborate with machine learning, applied science, full-stack product, platform, security, and data engineering teams.
Requirements
- At least three years of professional software engineering experience with ownership of production backend systems.
- Strong understanding of asynchronous or distributed systems, including queues, concurrency, state, storage, idempotency, partial failure, and recovery.
- Experience with systems where outputs can be materially wrong despite healthy services.
- Ability to define invariants and use reconciliation, control totals, diffs, replay, goldens, shadow paths, or independent sources to verify correctness.
- Experience designing versioned data and artifact contracts and migrating them safely in live systems.
- Ability to debug from evidence across system boundaries and turn incidents into durable improvements.
- Ability to prioritize technical work based on operator and customer consequences.
- Fluency with modern AI engineering tools and the ability to apply deterministic guardrails and human review.
- Clear communication across product, ML, data, platform, security, and customer-facing teams.
- Bonus experience with large-scale batch processing, workflow orchestration, event-driven systems, data movement, schema evolution, manifests, lineage, CDC, migrations, reindexing, backfills, sensitive or multi-tenant data, auditability, quarantine, or fail-closed release paths.
- Bonus experience combining deterministic checks, synthetic fixtures, offline replay, model-based judges, and human review.
- Experience with Python, AWS, Airflow, Batch, SQS, S3, DynamoDB, PostgreSQL, or comparable systems is a bonus.
Benefits
- Small-company environment with outsized responsibility and rapid growth.
- Opportunity to work with important companies and contribute to a category-defining business from the ground floor.
- The role uses AI tools as part of the daily engineering workflow.
Tech Stack
Categories
BackendData Engineering
About Sunset
We help tech companies shut down. From state withdrawals to liquidations, we save founders thousands of dollars, hundreds of hours, and countless headaches when it comes to winding down their operations.