Research Engineer - Pretraining Data
Prime Intellect5 hours ago
Remote, United States or San Francisco, CA, USAMid Level
H1B sponsor
Responsibilities
- Find, evaluate, and collect training data from web, code, and technical sources.
- Build scalable pipelines for extraction, cleaning, deduplication, filtering, and conversion of raw data into training-ready corpora.
- Develop data-quality, coverage, and contamination checks and use them to improve collection and processing.
- Collaborate with researchers on pretraining and midtraining data mixtures and sampling strategies.
- Prepare and evaluate datasets for long-context learning, reasoning, coding, and synthetic-data use cases.
- Run controlled data experiments and use model evaluations to determine which sources, mixtures, and processing choices to scale.
- Make large-scale data processing efficient, reproducible, recoverable, and traceable through source provenance and dataset versioning.
Requirements
- Strong Python programming skills and a record of building reliable data-processing or distributed systems.
- Experience working with large datasets and managing storage, compute, throughput, and reproducibility tradeoffs.
- Ability to diagnose data-quality issues and translate research goals into measurable checks and experiments.
- Ability to own an evolving technical problem end to end and collaborate closely with researchers.
- Relevant experience may include web crawling, document extraction, code-data collection, corpus curation, deduplication, text classification, quality scoring, synthetic data, contamination analysis, data-mixture experiments, curriculum learning, or measuring dataset effects on model performance.
- Preferred experience includes preparing datasets for language-model pretraining, continued pretraining, or midtraining; using Spark, Ray, or comparable distributed processing systems; working with multilingual, mathematical, scientific, or repository-level code data; and understanding tokenization, training data loaders, and large model-training runs.
Benefits
- Flexible work arrangements with the option to work remotely or in person at the San Francisco offices.
- Visa sponsorship and relocation assistance for international candidates.
- Quarterly team off-sites, hackathons, conferences, and learning opportunities.
- Opportunity to work with a mission-driven team building open frontier AI infrastructure.
Tech Stack
Categories
Data EngineeringML Engineering
About Prime Intellect
Prime Intellect builds a full-stack platform for training frontier AI models, giving enterprises access to distributed compute and agentic training infrastructure to develop their own systems. The company operates as a platform and open research lab, with customers leveraging its infrastructure rather than building it in-house. Headquartered in San Francisco and privately held, it has raised a Series A round backed by investors including Founders Fund and NVIDIA.