2 months ago
Bengaluru, IndiaSenior
Responsibilities
- Design, develop, deploy, and operate generative AI applications and the organization’s AI platform.
- Maintain LLM infrastructure, including a LiteLLM-based gateway with model routing, fallbacks, caching, cost tracking, FinOps, and guardrails.
- Build LLM observability capabilities for prompt and response logging, token and cost attribution, latency tracking, failure clustering, hallucination detection, and input drift monitoring.
- Develop and test AI agents and agentic workflows with tool use, orchestration, error handling, and human-in-the-loop mechanisms.
- Integrate and manage MCP servers connecting AI applications with Slack, Jira, databases, and internal APIs.
- Design evaluation frameworks using golden datasets, LLM-as-judge patterns, regression testing, and quality metrics.
- Support vector database infrastructure, ingestion pipelines, chunking strategies, retrieval measurement, and AI platform integration.
- Maintain Terraform infrastructure, GitHub Actions CI/CD pipelines, and AWS cloud resources.
- Collaborate with Data Science, Product, and Engineering teams to improve AI platform capabilities and developer experience.
Requirements
- 5+ years of professional experience in software engineering, platform engineering, DevOps, or AI/ML infrastructure.
- Hands-on production experience with LLM APIs, prompt design, function and tool calling, streaming, and structured outputs.
- Experience with LLM orchestration and gateway tools such as LiteLLM, LangChain, or similar frameworks.
- Experience managing, configuring, or supporting enterprise search or low-code/no-code AI platforms such as Glean or Gemini Enterprise Apps.
- Experience building or operating AI agents and agentic workflows using LangGraph, CrewAI, or custom implementations.
- Working knowledge of LLM application evaluation, including automated test suites, LLM-as-judge methods, golden datasets, or A/B comparison infrastructure.
- Proficiency in Python and experience working across backend services, APIs, and scripting.
- Experience with AWS and infrastructure-as-code tools such as Terraform.
- Experience building CI/CD pipelines with GitHub Actions, Azure DevOps, or Jenkins.
- Preferred experience with MCP, LLM observability tools, RAG architectures, prompt injection testing, LLM security, guardrails, AWS Lambda, semantic caching, model routing, and LLM cost optimization.
- Preferred experience in SaaS or enterprise software environments and strong troubleshooting and cross-functional communication skills.
Benefits
- Hybrid work arrangement requiring a minimum of two days per week in the office, with frequency subject to team and business needs.
- Docusign provides reasonable accommodations for qualified individuals with disabilities and for religious accommodations during the application process.
- The company emphasizes equal opportunity, open collaboration, and an inclusive workplace.
Categories
About DocuSign
DocuSign builds e-signature, contract lifecycle management, and agreement workflow software used by businesses to prepare, sign, and manage contracts. It sells cloud-based subscriptions and APIs that integrate with systems like Salesforce, Microsoft, and Google to automate document workflows and compliance. Founded in 2003 and headquartered in San Francisco, DocuSign is a public company on NASDAQ with over 1.5 million customers in more than 180 countries.