Senior Software Engineer II - LLM Observability
Honeycomb.io16 hours ago
Remote, United StatesSenior
Base Salary
$183k - $206k/yr
Responsibilities
- Own end-to-end development of AI observability capabilities from ingestion through product visualizations.
- Architect and evolve data models and infrastructure for observing AI agents and their ecosystems.
- Build systems that surface AI-specific signals including cost, token usage, errors, and anomalies.
- Collaborate with product managers and designers to deliver intuitive customer workflows.
- Teach best practices for understanding AI system behavior across conversations and workloads.
- Iterate through experimentation and fast feedback loops while sharing results and learnings.
- Participate in the on-call rotation after approximately 90 days.
Requirements
- Significant software engineering experience with a record of shipping and owning features end to end.
- Experience shipping customer-facing features at a SaaS company rather than only internal tooling.
- Full-stack proficiency with backend development in Go and/or Python and frontend development in TypeScript and React.
- Experience working closely with product managers and designers.
- Regular hands-on use of AI tools in day-to-day engineering work.
- Experience being on call.
- Ability to navigate ambiguity and shifting requirements in a fast-moving, experimentation-oriented environment.
- Preferred: experience building LLM- or agent-powered features in production.
- Preferred: hands-on experience with observability tooling such as OpenTelemetry.
Benefits
- Fully distributed and remote work arrangement.
- Generous equity with an employee-friendly stock program.
- Unlimited paid time off.
- Home office, co-working, and internet stipend.
- Full benefits coverage for employees, with additional dependent coverage available.
- Up to 16 weeks of paid parental leave regardless of path to parenthood.
- Annual development allowance.
Tech Stack
Categories
About Honeycomb.io
Honeycomb.io builds a SaaS observability platform for software engineering, SRE, and DevOps teams to debug and operate distributed systems using tracing, event data, and high-cardinality queries. Its product integrates with OpenTelemetry and supports SLOs, instrumentation, and production debugging workflows. Founded in 2016 and headquartered in San Francisco, the company is privately held and counts customers such as Slack, HelloFresh, LaunchDarkly, and Vanguard.