20 days ago
Base Salary
$200k - $280k/yr
Responsibilities
- Use evaluation results and production traces to improve the agent harness, model routing, prompts, retrieval, quality, latency, and cost.
- Post-train LLMs using supervised fine-tuning and preference- or reward-based optimization, including generating training data.
- Build RL environments for enterprise workflows with evalsets, harness support, and LLM-based verifiers.
- Improve evalsets, LLM judges, and agentic verifiers, and calibrate them against human graders.
- Partner with product, platform, and research teams to deliver quality improvements in production.
Requirements
- 5+ years of industry experience in AI or Machine Learning Engineering.
- Hands-on experience post-training LLMs and adapting open-weight models to specialized tasks.
- Experience building evaluations for AI assistants and agent systems, including evalsets, LLM-as-a-judge verifiers, and calibrated offline and online quality signals.
- Strong coding skills in Python and/or Go; TypeScript is a plus.
- Proficiency in an ML framework and a track record of shipping production ML systems.
- Product-minded approach and ability to turn ambiguous problems into concrete experiments.
Benefits
- Hybrid work arrangement requiring four days per week in the Mountain View or San Francisco office.
- Medical, vision, and dental coverage, generous paid time off, and a 401(k) plan.
- Home office improvement stipend plus annual education and wellness stipends.
- Regular company events and healthy lunches daily.
- Certain roles may be eligible for variable compensation, equity, and benefits.
Tech Stack
Categories
About Glean
Glean.ai is Intelligent AP. We leverage untapped invoice data to surface relevant context, actionable insights, and savings recommendations that power great spend decisions.