Software Engineer, ML Infrastructure
Arc Institute1 day ago
Base Salary
$191k - $237k/yr
Responsibilities
- Own the backend of Arc’s Model Performance Dashboard and define how virtual cell model performance is measured.
- Design and build evaluation tools that let researchers compare models and measure the impact of each dataset version.
- Redesign and operate the single-cell data pipeline that creates training sets, including agent-based dataset discovery, processing, curation, and automated quality checks.
- Own lab-in-the-loop infrastructure that connects model performance results to experiments designed by Arc’s science leads.
- Build or select validation frameworks, tests, evaluation systems, QC gates, and policies for agent-written code and applications.
- Own the train-and-evaluate CI system, support multi-node training, diagnose distributed-training failures, and improve throughput and GPU utilization on Arc-operated clusters.
- Work directly with ML researchers, scientists, and stakeholders to define requirements and set technical direction.
Requirements
- Bachelor’s degree with 8 years of relevant experience or Master’s degree with 6 years of relevant experience.
- Experience owning a production data or ML platform end to end, including pipelines, storage, versioning, and technical direction.
- Experience designing and building systems with AI tools and agents and making their output verifiable through evaluations, QC gates, or tests.
- Ability to learn unfamiliar technical domains well enough to make sound design decisions and work directly with stakeholders without an intermediary PM.
- Experience improving others through code review, pairing, and documenting engineering patterns.
- Experience diagnosing and fixing performance issues in Python or PyTorch systems using profilers and measurements of GPU utilization, memory, and data loading.
- Preferred: experience with single-cell or perturbation data, including AnnData or h5ad.
- Preferred: experience building or running model evaluations, benchmarks, evaluation harnesses, and PyTorch/GPU inference.
- Preferred: experience with multi-node distributed training using DDP, FSDP, NCCL, or SLURM, or improving training throughput.
- Preferred: experience leading an effort with one or two other engineers.
Benefits
- Base salary is accompanied by an annual discretionary bonus and eligibility for the Institute’s Compound Cash Grant Program, which provides multi-year cash grants paid in semi-annual installments.
- This is an onsite role requiring work from the Palo Alto office at least three days per week.
- The role offers direct collaboration with ML researchers, scientists, and experimental biology teams on Arc’s virtual cell initiatives.
Categories
Data EngineeringML Engineering
About Arc Institute
Arc Institute is an independent nonprofit research organization at the intersection of AI and biology, focused on understanding complex diseases and advancing new technologies. It operates investigator labs and Technology Centers that develop genome engineering, multiomics, cellular models, and machine-learning systems, including efforts like the Virtual Cell and Alzheimer’s Disease Initiatives. Founded in 2021 and headquartered in Palo Alto, it works in close partnership with Stanford, UC Berkeley, and UCSF.