20 hours ago
Responsibilities
- Build infrastructure for agent runtime, evaluation, quality measurement, and model optimization.
- Create tooling to identify evaluation strengths, coverage gaps, and unsupported user questions.
- Design signal pipelines that surface user frustration, agent failures, and quality issues.
- Define a HubSpot-specific AI benchmark using real HubSpot workloads.
- Evaluate new models across agents to improve quality, reduce cost, and support model routing.
- Make fine-tuning and task-specific optimization more repeatable for HubSpot AI use cases.
- Influence technical direction across infrastructure, product, backend, and ML teams.
Requirements
- Deep experience with production machine learning, LLM, or AI infrastructure.
- Experience building systems at scale where reliability, latency, quality, and cost matter.
- Experience with evaluation frameworks, model serving, fine-tuning, signal extraction, experimentation, feedback loops, or model optimization.
- Ability to think in platforms rather than one-off solutions and work across multiple technical teams.
- Ability to influence senior engineers and leaders through technical credibility and clear judgment.
- Comfort working on ambiguous, foundational problems where solutions still need to be invented.
Benefits
- HubSpot offers benefits and perks intended to support employee growth.
- Engineering employees must attend a regional HubSpot office for in-person onboarding, with flexibility to work remotely or from an office afterward.
- Employees may attend in-person events such as Product Group Summits and other gatherings.
- Accommodation support is available for candidates with travel limitations or other needs.
Categories
About HubSpot
HubSpot builds a cloud-based CRM platform with marketing automation, sales, customer service, content management, and operations tools for businesses, especially SMBs. It sells freemium and subscription SaaS plans with add-ons and a marketplace of integrations. Founded in 2006 and headquartered in Cambridge, Massachusetts, the public company (NYSE: HUBS) reports serving more than 250,000 customers globally.