9 hours ago
Bengaluru, IndiaSenior
Responsibilities
- Design, develop, deploy, and operate generative AI applications for enterprise problems.
- Contribute to the AI platform spanning LLM infrastructure, agent systems, Glean, Gemini Enterprise Apps, and Claude Cowork.
- Build and maintain a LiteLLM-based LLM gateway with model routing, fallbacks, caching, cost tracking, FinOps, and guardrails.
- Develop LLM observability for prompt and response logging, token and cost attribution, latency, failure modes, hallucinations, and input drift.
- Build and iterate on AI agents and agentic workflows with tool use, orchestration, error handling, and human-in-the-loop mechanisms.
- Integrate MCP servers with tools and data sources such as Slack, Jira, databases, and internal APIs.
- Create evaluation frameworks, golden datasets, LLM-as-judge systems, regression tests, and quality reporting for LLM applications.
- Support vector database infrastructure, ingestion pipelines, chunking, retrieval measurement, and AI platform integration.
- Maintain Terraform infrastructure, GitHub Actions CI/CD pipelines, and AWS cloud resources.
- Collaborate with Data Science, Product, and Engineering teams to improve AI application developer experience.
Requirements
- 5+ years of professional experience in software engineering, platform engineering, DevOps, or AI/ML infrastructure.
- Hands-on production experience with LLM APIs, prompt design, function and tool calling, streaming, and structured outputs.
- Familiarity with LiteLLM, LangChain, or similar LLM orchestration and gateway frameworks.
- Experience managing, configuring, enabling, or supporting enterprise AI platforms such as Glean or Gemini Enterprise Apps.
- Experience building or operating AI agents and agentic workflows using LangGraph, CrewAI, or custom implementations.
- Working knowledge of LLM evaluation methods including automated test suites, LLM-as-judge, golden datasets, and A/B comparison infrastructure.
- Proficiency in Python and experience with backend services, APIs, and scripting.
- Experience with AWS and infrastructure-as-code tools such as Terraform.
- Experience building CI/CD pipelines with GitHub Actions, Azure DevOps, or Jenkins.
- Preferred experience with Model Context Protocol and MCP servers.
- Preferred experience with LLM observability tools such as Arize, Braintrust, Datadog LLM monitoring, LangSmith, or custom tracing solutions.
- Familiarity with RAG architectures, embedding models, vector stores, retrieval strategies, and quality evaluation.
- Exposure to prompt injection testing, LLM security, guardrails, AWS Lambda, semantic caching, model routing, or LLM FinOps.
- Strong troubleshooting, debugging, communication, and cross-functional collaboration skills; SaaS or enterprise software experience is preferred.
Benefits
- Hybrid work arrangement requiring a minimum of two days per week in the office, with frequency potentially varying by team.
- Docusign provides reasonable accommodations for qualified individuals with disabilities and for religious accommodations during the application process.
Categories
About DocuSign
DocuSign builds e-signature, contract lifecycle management, and agreement workflow software used by businesses to prepare, sign, and manage contracts. It sells cloud-based subscriptions and APIs that integrate with systems like Salesforce, Microsoft, and Google to automate document workflows and compliance. Founded in 2003 and headquartered in San Francisco, DocuSign is a public company on NASDAQ with over 1.5 million customers in more than 180 countries.