12 hours ago
Base Salary
$225k - $325k/yr
Responsibilities
- Analyze production conversations, calls, traces, and customer feedback to identify failure patterns and prioritize improvements.
- Define quality dimensions, evaluation datasets, automated graders, human-review processes, and production metrics.
- Design reproducible experiments, offline evaluations, and controlled production rollouts to assess agent improvements and tradeoffs.
- Improve voice-agent turn-taking, interruption handling, response latency, transcription accuracy, speech quality, and conversational naturalness.
- Implement, deploy, and monitor agent-system improvements while owning reliability, cost, regressions, and quality outcomes.
- Create reusable datasets, evaluations, debugging tools, and release checks from production failures and experimental findings.
- At Staff level, shape evaluation and experimentation strategy and raise the team’s ability to diagnose and improve agent behavior.
Requirements
- Demonstrated experience improving substantial AI agent systems deployed to external customers, including explaining system complexity, experiments, implementation, and production impact.
- Strong experimental and analytical judgment, including baseline definition, comparison design, reasoning about noisy or confounded results, and learning from unsuccessful hypotheses.
- Strong software engineering ability, including analyzing conversational data with code and SQL, debugging complex agent behavior, and building maintainable production software.
- Practical understanding of agent systems, including interactions among models, instructions, context, tools, orchestration, quality, latency, reliability, and cost.
- Clear communication, ownership, ability to work through ambiguity, and effective collaboration with engineering and product teams.
- Experience improving production voice agents is strongly preferred, including diagnosing conversational timing, speech recognition and generation, audio quality, and end-to-end latency.
- Experience building simulation, replay, or evaluation tools for multi-turn agents is nice to have.
- Experience developing human-review workflows or calibrating automated evaluations against human judgment is nice to have.
- Experience at a strong startup or on a small team with substantial ownership is nice to have.
Benefits
- $225,000–$325,000 base salary plus meaningful early equity.
- Four days per week in the Flatiron office and one remote day, with flexibility when life comes up.
- $0 employee premiums for individual medical coverage on most plans, plus dental and vision insurance.
- 401(k) matching up to 4% of salary.
- Unlimited PTO and commuter benefits.
- Free lunches and dinners in the office.
- $100 per month gym membership stipend.
- Tech equipment of the employee’s choice.
- Full-time role on an early-stage startup team with substantial ownership and influence.
Tech Stack
Categories
About Revin
Revin builds an AI-driven, modular SaaS platform for home services, construction, and remodeling operators, covering scheduling, appointment management, inside sales, revenue capture, and marketing. Headquartered in New York, it sells to multi-location and private equity–backed enterprises; customers include Sears Home Services, Del Air, Long Home Remodeling, and Sage Home. Over $2.3B in customer revenue runs through Revin’s system.