2 days ago
Base Salary
$180k - $450k/yr
Responsibilities
- Research model architecture, optimization, and scaling methods to improve the capability and efficiency of large models.
- Design baselines and controlled experiments to evaluate architectural and optimization changes at larger scales.
- Develop training recipes covering learning-rate schedules, context lengths, token budgets, and compute allocation.
- Investigate mixture-of-experts architectures, hybrid attention, and long-context training.
- Diagnose loss spikes, divergence, and numerical issues while distinguishing modeling problems from infrastructure failures.
- Implement research ideas, profile training workloads, and debug model behavior and distributed training systems.
Requirements
- Hands-on experience training large models and balancing model quality, compute cost, and training stability.
- Strong experimental judgment and experience designing controlled experiments to distinguish competing hypotheses.
- Experience applying scaling laws to experiment design and evaluating extrapolations from smaller training runs.
- Experience implementing and debugging distributed training workloads across GPU clusters.
- Strong engineering skills in research implementation, training-workload profiling, and model-behavior debugging.
- A record of improving model quality or training efficiency through research, engineering, open-source contributions, or production work.
- Preferred experience includes mixture-of-experts routing, sparse architectures, long-context methods, data mixtures, curriculum, tokenizer design, kernel-level optimization, mixed-precision training, or efficiency work for constrained inference targets such as on-device systems.
Benefits
- Full-time position with a US base salary range of $180,000–$450,000 annually.
Categories
AI Research
