7 hours ago
Base Salary
$275k - $350k/yr
Responsibilities
- Own the long-term architecture for direct-push ingestion, scheduled third-party pulls, non-telemetry data, and transformation pipelines.
- Define storage, query, retention, cost, and performance strategies across time-series, columnar/OLAP, and lakehouse systems.
- Design query interfaces, freshness guarantees, performance commitments, and sandboxing for AI-agent workloads.
- Build data foundations for AI research, including investigation history, evaluation datasets, and derived signals.
- Lead orchestration and transformation systems producing materialized views, rollups, entity graphs, and cached LLM-based enrichment.
- Define and meet reliability, availability, data-freshness, query-latency, monitoring, rollout, and cost objectives.
- Design customer-data controls for redaction, tenant isolation, and data residency.
- Lead architecture reviews, customer deployments, multi-quarter migrations, and platform transformations through adoption and deprecation.
- Mentor senior engineers and establish standards and reference implementations adopted across teams.
Requirements
- Typically 10+ years of experience building and operating production data infrastructure, including setting architecture for platforms used by multiple teams.
- Deep distributed-systems expertise in streaming data, analytical storage, query engines, and scalable multi-tenant systems.
- Experience with Kafka or equivalent, ClickHouse, Doris, Druid, Pinot, and data lakehouse technologies.
- Strong systems programming experience in Rust or comparable experience in Go, plus comfort with Python.
- Experience shipping on Kubernetes, Helm, Terraform, and AWS, ideally in BYOC, on-premises, or regulated customer-cloud environments.
- Track record of leading multi-quarter migrations and platform transformations through adoption and deprecation.
- Regular production coding experience and the ability to discuss personally shipped work.
- Strong judgment regarding performance, cost, durability, simplicity, technical tradeoffs, and system operating costs.
- Clear technical writing, architecture-review judgment, and ability to align teams without formal authority.
- Preferred experience with observability, logging, APM, or SIEM product backends.
- Preferred fluency with OpenTelemetry/OTLP and observability platforms or APIs including Datadog, Splunk, Elastic, Prometheus, and Dynatrace.
- Preferred experience with Airflow, Dagster, Temporal, data-pipeline frameworks, LLM-powered data pipelines, embeddings, data governance, regulated industries, or open-source data infrastructure.
Benefits
- Competitive compensation, startup equity, health insurance, fertility benefits, a technology setup stipend, and flexible time off.
- In-office snacks, team happy hours and outings, an annual company offsite, and built-in collaboration time.
- Full-time, fully in-office role five days per week in New York near Madison Square Park.
Tech Stack
Categories
Data Engineering