3 months ago
Remote, WorldwideSenior
Responsibilities
- Design, build, and operate large-scale ingestion systems for structured and unstructured multimodal data.
- Own ingestion pipelines end to end, including data landing, validation, processing, tracking, provenance, and downstream availability.
- Develop modality-specific processing for medical imaging, audio, video, text, and other source data.
- Build parsers, validators, normalization logic, reusable transforms, internal tooling, and platform capabilities.
- Use distributed, parallel, batch, and other compute models to process high-volume workloads.
- Optimize ingestion and processing systems for reliability, cost, speed, throughput, and scalability.
- Build data-quality checks, observability, debuggability, and operational reliability into the ingestion layer.
- Handle sensitive and regulated data, including PHI, de-identification, security, and compliance requirements.
- Partner with product, Data Lab, and partner engineering teams on new modalities and source-system requirements.
- Lead design for new modalities and scaling challenges and identify architectural and tooling improvements.
Requirements
- 5+ years building and operating production backend or data systems, including hands-on data processing at scale.
- Hands-on experience designing and running large-scale data pipelines and distributed data processing systems.
- Strong programming skills in Python and strong proficiency with AWS.
- Ability to work with messy, varied, high-volume data and ambiguous problem spaces.
- Experience with medical imaging such as DICOM, text, audio, or video processing is preferred.
- Experience with sensitive or regulated data environments, HIPAA, healthcare compliance, or PHI handling is preferred.
- Experience with streaming systems or workflow orchestration such as Airflow or Dagster is preferred.
- Experience with GCP and Azure is preferred.
- Familiarity with ML, NLP, or LLM-based systems, including embeddings and fine-tuning, is preferred.
- Prior startup experience as a founding or early engineer is preferred.
Tech Stack
Categories
BackendData Engineering
