AI Research Engineer (Pre-training - LLM & Multi-Modal)

3 months ago

Remote, WorldwideSenior

Responsibilities

Conduct foundational large-scale pre-training for LLMs and multimodal models using text, vision, audio, and other modalities.
Design, prototype, and scale model architectures, tokenizers, and cross-modal alignment layers.
Source, filter, and curate large textual and multimodal datasets and establish efficient data pipelines.
Execute experiments, analyze results, and refine training methodologies for performance and token efficiency.
Debug model efficiency, computational performance, and multimodal alignment stability issues during long training runs.
Advance distributed training systems for scalability and hardware efficiency.

Degree in Computer Science or a related field.
Hands-on experience contributing to large-scale LLM or multimodal pre-training on distributed systems with thousands of NVIDIA GPUs.
Practical experience with large-scale distributed training frameworks, libraries, and tools.
Deep knowledge of state-of-the-art transformer and non-transformer modifications for model intelligence, efficiency, and scalability.
Strong expertise in PyTorch and Hugging Face libraries, including model development, continual pre-training, and deployment.
Strong AI research and development track record; a PhD in NLP, Machine Learning, or a related field and publications at A* conferences are preferred.

201-500 employees