1 month ago
Pune, IndiaMid Level
Responsibilities
- Build and maintain operational tooling for LLM-powered systems, including prompt and pipeline versioning, evaluation harnesses, and CI/CD workflows.
- Implement monitoring and observability for production LLM systems, including latency, token usage, cost, output quality, and drift tracking.
- Design and run automated evaluation suites to identify regressions, hallucinations, and quality degradation.
- Manage RAG pipelines and vector store infrastructure while keeping retrieval sources fresh, accurate, and performant.
- Implement content filtering, PII redaction, access controls, and other safety and compliance guardrails.
- Own LLM cost governance through caching, model routing, and usage reporting.
- Collaborate with Product Managers and Senior Engineers to define operational requirements and build reliable systems.
- Participate in sprint planning, technical grooming, and retrospective discussions.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field.
- 4+ years of experience in a software engineering role with a solid background in backend engineering and DevOps fundamentals.
- Hands-on experience operating or supporting LLM-powered systems in production through APIs, RAG pipelines, frontier models, open-weight models, or fine-tuned models.
- Proficiency in a backend language such as Python, Java, TypeScript, or Go.
- Familiarity with Docker, Kubernetes, APIs, SQL and NoSQL databases, automated tests, cloud platforms, infrastructure-as-code, and CI/CD tooling.
- Experience with at least one of AWS, GCP, or Azure across managed and self-hosted services.
- Preferred experience with LangFuse, LangSmith, Weights & Biases-style evaluation frameworks, Pinecone, Weaviate, pgvector, LangGraph, LiteLLM, OpenRouter, Vertex AI, Bedrock, and MCP implementation or governance.
- Understanding of prompt management, model access management, deterministic guardrails, evaluations, data privacy, data residency, and enterprise AI compliance.
- Ability to debug production issues involving non-deterministic systems and collaborate across engineering, product, and strategy functions.
Benefits
- Work at the operational core of Coursera’s AI transformation on reliable, safe, and cost-effective production AI systems.
- Build hands-on expertise in a growing LLMOps engineering discipline.
- Join a supportive, innovative team with a culture of continuous learning and improvement.
- Contribute to a mission-driven company focused on global access to education and upskilling.
- Job location is Pune.
Tech Stack
AWSAzureDockerGitHub ActionsGoGoogle Cloud PlatformJavaJenkinsKubernetesPythonSQLTerraformTypeScript
Categories
About Coursera
Coursera builds an online learning platform for individuals, enterprises, and academic institutions, offering courses, Professional Certificates, and online degrees from university and industry partners. Its revenue comes from paid courses and subscriptions, enterprise contracts, and tuition-sharing for degree programs. Founded in 2012 by Andrew Ng and Daphne Koller and headquartered in Mountain View, it is publicly traded on the NYSE (COUR). Coursera has combined with Udemy to form a single, larger skills platform.