4 months ago
Remote, United States or Oakland, CA, USAMid Level / Senior
Base Salary
$140k - $200k/yr
Responsibilities
- Build a comprehensive auto-eval platform that runs efficiently and is user-friendly.
- Optimize interfaces for ML engineers, product managers, and customers.
- Ensure evaluations accurately reflect decision-making support in pharma.
- Provide statistical tests and confidence intervals for reliable results.
- Mentor the evals engineering intern and collaborate with the evals team.
Requirements
- At least 3 years of experience as a professional software engineer.
- Demonstrated experience building complex backend systems.
- Aptitude and interest in evaluating decision-making in pharma.
- Knowledge of statistics for calculating power and confidence intervals.
- Experience with advanced Python, including asyncio/trio and parallel processing.
Benefits
- Flexible work environment with options for remote or hybrid work.
- Fully covered health, dental, vision, and life insurance.
- Flexible vacation policy with a minimum of 20 days/year.
- 401K with a 6% employer match.
- Quarterly AI Experimentation & Learning budget of $1,000.