8 hours ago
Bengaluru, IndiaEntry Level
Responsibilities
- Build and run evaluations for LLM-powered chatbot systems.
- Design and calibrate LLM-as-judge scorers with subject matter experts against human ground truth.
- Contribute to conversation simulation and replay infrastructure.
- Assist with real-time observability tooling for chatbot performance degradation.
- Analyze production conversations and surface insights for chatbot improvements.
- Work with ML engineers and analysts on prompt mutations, model changes, and evaluation pipelines.
Requirements
- Currently pursuing a bachelor's or master's degree in computer science, software engineering, or a related field from an Indian university.
- Familiarity with Python and comfort working with APIs, data pipelines, or scripting.
- Curiosity about large language models, AI evaluation, and reliable generative AI systems.
- Basic NLP knowledge or hands-on LLM experience through projects or coursework is preferred.
- Understanding of probability theory and stochastic systems is preferred.
- Strong analytical thinking, communication, self-direction, and follow-through.
Benefits
- Six-month contract engagement starting in January 2027.
- Bengaluru, India location with two days per week in the office.
- Work with ML engineers and analysts on chatbot evaluation and improvement systems.
Tech Stack
Categories
About EarnIn
EarnIn builds a mobile app for earned wage access and related financial tools for U.S. workers living paycheck to paycheck. It lets users access a portion of accrued wages before payday, plus features like credit monitoring, automated savings, low-balance alerts, and a debit card, with no mandatory fees or interest. Founded in 2012 and headquartered in Mountain View, California, the privately held company issues certain banking products via Evolve Bank & Trust.