2 months ago
Bengaluru, IndiaMid Level
H1B Sponsor
Responsibilities
- Design and build large-scale evaluation pipelines for assistant and agent quality across real-user and synthetic workflows.
- Evaluate frontier model releases and open-source model drops and create quality signals for regressions, tradeoffs, and launch readiness.
- Build agent observability infrastructure, including trace enrichment, durable telemetry pipelines, dashboards, and debugging workflows.
- Own backend systems from architecture and design documentation through production rollout, reliability, monitoring, and iteration.
- Partner with product, ML, and infrastructure engineers to make evaluations a core part of AI feature delivery.
- Connect evaluation results, customer feedback, regression analysis, and engineering workflows to product improvements.
- Build scalable, secure, permissions-aware, fast, and cost-efficient systems in modern cloud-native environments.
- Mentor engineers and contribute to technical design and engineering culture.
Requirements
- 4+ years of software engineering experience building backend systems, infrastructure, distributed systems, or data platforms.
- Strong coding skills in Go, Python, Java, C++, or similar languages, with an emphasis on reliability, scale, and well-tested components.
- Experience working with distributed data pipelines, production services, observability systems, or cloud-native infrastructure.
- Analytical rigor and attention to whether metrics reflect real user experience.
- Ability to work in customer-focused, cross-functional environments and take on high-impact work.
- Strong commitment to quality in both software systems and AI products.
- Experience with LLM applications, evals, tracing, data warehouses, workflow orchestration, or ML infrastructure is a strong plus.
Benefits
- In-person work arrangement in Bangalore, India
- Potential eligibility for variable compensation, equity, and benefits
- Inclusive and diverse workplace commitment
About Glean
Glean.ai is Intelligent AP. We leverage untapped invoice data to surface relevant context, actionable insights, and savings recommendations that power great spend decisions.