Distributed Spectrum

Technical Lead, Data Engineering

Distributed Spectrum
Apply
6 hours ago

Base Salary

$160k - $200k/yr

Responsibilities

  • Architect and build a data lakehouse with ingestion pipelines, open table formats, partitioning, compaction, cataloging, and governance.
  • Design, operate, tune, and scale vector database infrastructure for embedding storage, similarity search, and retrieval.
  • Build data management capabilities for lifecycle and retention, lineage, dataset versioning, quality monitoring, and cost management across petabyte-class storage.
  • Develop ML-focused feature and embedding pipelines, training-set assembly, evaluation datasets, and low-latency retrieval for agents.
  • Establish schema and data-contract standards and build self-service tooling for researchers and engineers.
  • Own production data-pipeline reliability, performance, observability, and failure handling.
  • Mentor engineers and lead technical design across the data domain.

Requirements

  • 4–6+ years of software or data engineering experience, including several years designing and operating large-scale data platforms in production.
  • Deep experience with lakehouse architectures and technologies such as Apache Iceberg, Delta Lake, Hudi, Parquet, Spark, Flink, Trino, or DuckDB.
  • Hands-on experience with vector databases and embedding retrieval, including indexing trade-offs and scaling.
  • Strong experience with AWS or another major cloud provider, including object storage, managed data services, compute, and cost optimization at scale.
  • Fluency in Python and SQL and experience with orchestration tools such as Airflow, Dagster, Prefect, or Step Functions.
  • Experience designing data architectures for AI/ML workloads, including training data pipelines, feature stores, dataset versioning, or retrieval systems.
  • Strong understanding of data modeling, consistency, and batch-versus-streaming trade-offs.
  • Must currently hold or be eligible to obtain and maintain a U.S. government security clearance.
  • Experience with high-volume streaming ingestion using Kafka, Kinesis, or Pulsar is preferred.
  • Experience with time-series, geospatial, signal, or sensor data is preferred.
  • Familiarity with data governance and security requirements in regulated environments is preferred.
  • Experience with Rust, Go, or C++ for performance-critical data paths is preferred.

Benefits

  • Above-market salary, equity, and benefits package.
  • Early Series A equity.
  • Excellent health, dental, and vision coverage.
  • 401(k) match of up to 4% of salary.
  • Flexible PTO.
  • Daily office lunches in NYC.
  • ITAR eligibility requirements apply: candidates must be U.S. citizens, lawful permanent residents, protected individuals, or eligible to obtain required authorizations.
  • The role is based in an NYC office; daily office lunches are provided.

Tech Stack

Categories

Data Engineering
Distributed Spectrum

About Distributed Spectrum

11-50 employees

Distributed Spectrum builds software and sensor systems for radio spectrum intelligence, combining signal processing, machine learning, and embedded computing for defense and government operators. Its products collect and analyze RF data and run trained models on low-power devices to detect and interpret critical signals in the field. Founded in 2020 and headquartered in New York, the company is privately held and venture-backed.

Contact me