7 months ago
Pune, IndiaSenior
Responsibilities
- Build and integrate AI-driven features using OpenAI, Azure OpenAI, Anthropic, and Gemini on Vertex AI.
- Design tool-using agents, multi-agent workflows, retrieval-augmented generation, search workflows, and context-engineering systems.
- Create evaluation frameworks, golden datasets, adversarial cases, regression tests, automated evaluation pipelines, release gates, and online quality metrics.
- Develop scalable backend services and APIs, integrate AI pipelines into cloud and event-driven architectures, and build internal evaluation and deployment tooling.
- Implement tracing, observability, prompt versioning, cost tracking, reliability controls, guardrails, fallback logic, and safe tool execution.
- Partner with product managers, designers, the Chief Architect, and domain experts to prototype and improve AI-first solutions.
- Document system behavior, failure modes, operational playbooks, compliance considerations, and responsible AI practices.
- Ship a production agent workflow within the first 90 days with observability, offline evaluations, regression gates, cost and latency monitoring, and documented fallbacks.
Requirements
- Strong software engineering background, preferably with Python, and experience shipping backend services.
- Deep hands-on experience building production agentic LLM systems, including agent loops, tool interfaces, planning, replanning, memory, state, and failure handling.
- Strong context-engineering skills covering retrieval, routing, grounding, context budgeting, and long-context tradeoffs.
- Strong evaluation experience with golden datasets, regression gating, automated evaluation pipelines, and online monitoring.
- Practical experience with LLM APIs such as OpenAI, Azure OpenAI, Anthropic, and Gemini, plus AI orchestration frameworks.
- Experience shipping an LLM or agent system in production and explaining its failure modes, evaluations, cost and latency improvements, and monitoring approach.
- Excellent debugging, systems thinking, problem decomposition, ownership, and ability to work in ambiguous environments.
- Preferred experience with AWS or GCP, microservices, event-driven systems, OpenTelemetry, Datadog, Honeycomb, Langfuse, Braintrust, HumanLoop, W&B Weave, Temporal, Celery, or Airflow.
- Preferred experience with enterprise AI permissions, auditability, compliance, PII handling, prompt-injection defenses, and sandboxed tool execution.
Benefits
- Opportunity to build core AI capabilities that directly affect users and product strategy.
- Work on applied, real-world agentic AI systems without requiring model training.
- High ownership, rapid iteration, cross-functional collaboration, competitive compensation, and opportunities for rapid advancement.
Tech Stack
Categories
About Onit
Onit builds enterprise legal operations software used by corporate legal departments and law firms to manage contracts, matters, e‑billing, legal holds, and related workflows. Its products are delivered as SaaS with implementation and integration services, emphasizing process automation and AI‑assisted review and approval. The company is privately held, headquartered in Atlanta, Georgia, and operates globally with additional offices worldwide.