10 months ago
Bengaluru, IndiaMid Level
Responsibilities
- Build and maintain agentic workflows and datasets for training and evaluation.
- Run systematic LLM evaluations, define performance metrics, track regressions, and monitor model quality.
- Experiment with prompting, fine-tuning datasets, retrieval strategies, model configurations, and guardrails.
- Develop internal tools and automated pipelines for evaluation, annotation, regression checks, and model monitoring.
- Create feedback mechanisms that convert real user interactions and customer insights into model and product improvements.
- Ensure reliable, consistent agent behavior at production scale and debug systems spanning prompts, tools, APIs, and model responses.
- Tune latency, turn-taking, and conversational naturalness for voice AI systems.
- Write reliable Python or JavaScript code for model pipelines, tools, and integrations.
- Collaborate across Engineering, Product, and Customer teams to align agent capabilities with user needs.
Requirements
- 2 –5 years of experience in software engineering building reliable, scalable systems.
- 1–2 years of experience working on AI-backed products, including LLM engineering, evaluations, prompting, or agent development.
- Hands-on experience with LLMs, prompt engineering, evaluation frameworks, or dataset creation.
- Strong understanding of prompting, fine-tuning, evaluation, guardrails, and AI system design.
- Solid software engineering skills with comfort using APIs and cloud environments.
- Experience building internal tools or automation pipelines.
- Strong customer-oriented communication, collaboration, attention to detail, and outcome focus.
- Bonus experience with voice AI, speech technologies, or real-time agent systems.
Benefits
- Work on healthcare AI where accuracy, trust, and reliable real-world workflows are critical.
- Join a small, fast-growing, engineering-first startup and collaborate with engineers globally.
- Opportunity to grow as an individual contributor or team leader and set personal goals.
Tech Stack
Categories
