Research Engineer - Midtraining

about 5 hours ago

Menlo Park, CA, USASenior

H1B Sponsor

Base Salary

$250k - $350k/yr

Responsibilities

Identify, process, and curate novel scientific data sources for large-scale model training.
Generate synthetic data to address gaps in scientific knowledge and reasoning.
Build evaluations correlated with downstream scientific task performance in collaboration with RL researchers, physicists, and chemists.
Develop and apply self-distillation and on-policy distillation techniques to improve model capability.
Design and run large-scale training experiments with supercompute engineers across thousands of GPUs.
Build tools to investigate how data choices affect model intelligence.

Experience training LLMs on curated mixes of trillions of tokens.
Experience with mid-training or pre-training at scale; big-lab experience is a strong plus.
Experience on a dedicated evaluations team supporting a large production training run.
Hands-on experience with self-distillation, on-policy distillation, or similar methods in a real training pipeline.
Ability to calculate scaling laws and compute-optimal hyperparameters.
Comfort working across data, evaluations, and training infrastructure.
Experience optimizing throughput and reliability for large-scale distributed training runs is an especially strong qualification.
Background in AI for science or training on specialized domain data such as protein or materials datasets is an especially strong qualification.
Experience tracking evaluations and driving interventions during a live large-scale training run is an especially strong qualification.
Bachelor's degree or similar experience.

11-50 employees

We're building AI scientists and the autonomous laboratories for them to operate. Join us: https://jobs.ashbyhq.com/periodic-labs