3 days ago
Remote, EMEA or Prague, CzechiaSenior
Responsibilities
- Enhance LoRA-based and full-parameter fine-tuning methods for large language and foundation models.
- Build model training and evaluation pipelines in JAX for inference optimization and speculative decoding.
- Investigate dense and mixture-of-experts architectures, autoregressive and parallel approaches, and scaling laws for resource allocation.
- Develop low-precision FP8 and NVFP4/MXFP4 training and inference methods for supervised fine-tuning and reinforcement learning.
- Train large models across multiple computational nodes and optimize distributed training performance.
- Apply strong Python software engineering practices, including version control, unit testing, and CI/CD.
- Collaborate with engineering teams and provide technical leadership on large-scale AI systems.
Requirements
- Profound understanding of the theoretical foundations of machine learning and reinforcement learning.
- Deep expertise in modern deep learning for language processing and generation.
- Experience training large models across multiple computational nodes.
- Understanding of neural network performance, sharding strategies, custom kernels, and hardware features.
- Strong software engineering skills, particularly in Python.
- Deep experience with modern deep learning frameworks, especially JAX.
- Proficiency with contemporary software engineering practices, including CI/CD, version control, and unit testing.
- Strong communication and leadership abilities.
- Experience with language models or related NLP technologies is preferred.
- Familiarity with MHA, RoPE, ZeRO/FSDP, Flash Attention, and quantization is preferred.
- Experience building large distributed systems or high-load web services, delivering products, or contributing open-source projects is preferred.
- Excellent English-language communication, writing, and articulation skills are preferred.
Benefits
- Competitive compensation.
- Career growth and learning opportunities.
- Flexibility and ownership.
- Collaborative and innovative culture.
- Opportunity to work on impactful AI projects.
- International environment with talented teams.
Tech Stack
Categories
About Nebius
The Nebius AI Cloud brings powerful full-stack infrastructure for AI developers and practitioners across startups, enterprises and science institutes to build and deploy generative AI applications and rapidly deliver scientific breakthroughs by training and running ML models within a secure, high-performance, and cost-optimized cloud environment.