MTS, Post-Training (Enterprise)
Bespoke Labs4 hours ago
Base Salary
$300k - $350k/yr
Responsibilities
- Post-train, fine-tune, and align open-weight and proprietary base models for complex enterprise domains and production deployment.
- Build custom evaluation suites, benchmarks, evaluation datasets, and calibrated LLM judges for subjective tasks.
- Curate production datasets using traces, human labeling, synthetic augmentation, and strict filtering.
- Track and mitigate regression risks across downstream performance and core model capabilities.
- Work directly with enterprise customers, product teams, and other stakeholders to translate requirements into model behavior and evaluation metrics.
- Deploy and optimize models for enterprise latency, cost, and privacy requirements.
- Own the post-training lifecycle from raw data through deployment, failure analysis, and production impact measurement.
Requirements
- Previous experience post-training at least one LLM and deploying it to real users in production.
- End-to-end ownership of benchmarks, evaluation datasets, and quality metrics for complex or subjective tasks.
- Deep understanding of model regression risks and techniques for preventing capability degradation.
- Experience collaborating with enterprise customers, product managers, or other non-ML stakeholders.
- Strong software and machine learning fundamentals, including modern post-training workflows, data processing pipelines, and production codebases.
- Ability to independently own the full post-training lifecycle.
- Preferred experience with conversational or task-oriented assistants, LLM judges, reward models, production data flywheels, and open-weight models such as Llama, Qwen, Mistral, or DeepSeek.
- Experience from forward-deployed engineering, founder, or early-stage startup environments is a plus.
Benefits
- Mountain View, California is preferred and San Francisco, California is onsite; remote work is considered.
- Base salary is $300,000–$350,000 USD per year, with a 25% performance-based bonus and equity.
- Health, dental, and vision coverage are provided.
- 401(k) is provided.
- Daily onsite lunch is provided.
- Visa sponsorship and relocation support are available.