about 3 hours ago
San Francisco, CA, USAMid Level / Senior
Base Salary
$200k - $350k/yr
Responsibilities
- Build the eval and optimization loop for Console’s core agents.
- Systematically improve agent behavior across various parameters.
- Fine-tune and evaluate specialist models for high-volume tasks.
- Work across the stack from traces to customer-facing AI behavior.
Requirements
- Strong technical background in software engineering or applied AI.
- Experience building evals for AI systems, including datasets and metrics.
- Practical understanding of modern model adaptation methods.
- Ownership mindset with a focus on driving projects end-to-end.
- Interest in staying updated with state-of-the-art research.
Benefits
- Comprehensive health, dental, and vision insurance.
- Unlimited PTO.
- 401(k) plan.
- Equity options.
- Daily meals provided in the office.