2 months ago
Base Salary
$200k - $350k/yr
Responsibilities
- Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, and reward modeling.
- Translate research and partner needs into hypotheses, experiments, evaluation plans, and production-quality implementations.
- Build evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.
- Run rapid iteration loops by prototyping, evaluating, interpreting results, and improving systems or products.
- Partner with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.
- Identify repeatable patterns and productize them into reusable software and platforms.
- Raise technical standards through design judgment, communication, code quality, and mentorship.
- Contribute through benchmarks, open-source tools, research, and technical writing when useful.
Requirements
- At least 3 years of demonstrated strength in post-training, fine-tuning, or model-evaluation work.
- Relevant experience may include reinforcement learning, supervised fine-tuning, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, or training environments.
- Strong Python skills and the ability to write clean, efficient, scalable software.
- Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows.
- Strong experimental judgment, including forming hypotheses, selecting metrics, diagnosing failures, and distinguishing signal from noise.
- Experience designing systems and making tradeoffs involving quality, scale, reliability, and reuse.
- Ability to operate with substantial ownership in an ambiguous, fast-moving environment.
- Collaborative communication and ability to work effectively with researchers, engineers, domain experts, and customers.
- Preferred experience includes large-scale ML training, inference, data, or evaluation systems; LLM or agent benchmarks; annotation or data-quality frameworks; reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems; published research; open-source contributions; technical leadership; or productizing research into reusable platforms.
Benefits
- Equity in a fast-growing company.
- 401(k) match, competitive compensation, and financial coaching.
- Paid parental leave, fertility benefits, and parental coaching.
- Medical, dental, vision, mental health support, and a $500 wellness stipend.
- A $2,000 learning stipend and ongoing development.
- Commuting support, free lunch, and gym access at the San Francisco office.
- Flexible PTO, 15 holidays, and 2 flex days.
- Team outings and referral bonuses.
- Full-time US employee benefits apply; San Francisco and Mountain View are preferred, with exceptional candidates considered in other locations.
Categories
AI ResearchML Engineering
About Handshake
Handshake builds a career network and recruiting platform that connects college students and recent grads with employers, sold as SaaS to universities and subscriptions/solutions to employers. Founded in 2014 and headquartered in San Francisco, it serves 1,600+ educational institutions and over 1 million employers. The company also operates Handshake AI, which partners with frontier AI labs on human data collection and evaluations for model training and post-training workflows.