AI Research Engineer (Pre-training - LLM & Multi-Modal)

3 months ago

Remote, WorldwideSenior

Responsibilities

Conduct foundational pre-training for LLMs and multi-modal models integrating text, vision, audio, and other modalities on distributed servers with thousands of NVIDIA GPUs.
Design, prototype, and scale architectures, tokenizers, and cross-modal alignment layers.
Source, filter, and curate large-scale textual and multi-modal datasets and establish data pipelines for pre-training.
Execute experiments, analyze results, and refine training methodologies for model performance and token efficiency.
Investigate and resolve bottlenecks in model efficiency, computational performance, and multi-modal alignment stability.
Advance distributed training systems to improve scalability and hardware efficiency.

Degree in Computer Science or a related field.
Hands-on experience contributing to large-scale LLM or multi-modal pre-training runs on distributed systems with thousands of NVIDIA GPUs.
Practical experience with large-scale distributed training frameworks, libraries, and tools.
Deep knowledge of state-of-the-art transformer and non-transformer modifications for intelligence, efficiency, and scalability.
Strong expertise in PyTorch and Hugging Face libraries, including model development, continual pre-training, and deployment.
PhD in NLP, Machine Learning, or a related field and a strong AI R&D publication record at A* conferences are preferred.

201-500 employees