11 days ago
Boston, MA, USAMid Level
Base Salary
$109k - $164k/yr
Responsibilities
- Design and refine system prompts, character behavior definitions, and guardrails for the AI character platform.
- Build evaluation frameworks for character fidelity, response quality, safety, adversarial testing, and regression testing.
- Architect and implement RAG systems using embedding strategies, vector retrieval, chunking, and relevance evaluation.
- Partner with creative, brand, and engineering teams to translate character personality, voice, canon, tone, and boundaries into AI system requirements.
- Manage prompt versioning, model evaluation, and consistency practices across provider and model updates.
- Integrate with and evaluate multi-provider LLM APIs, including OpenAI, Google Gemini, and Anthropic.
- Evaluate emerging prompt engineering, agentic design, and LLM evaluation techniques.
Requirements
- At least 3 years of experience building and shipping production LLM-powered systems.
- Deep expertise in prompt engineering, including system prompts, few-shot techniques, chain-of-thought, multi-turn conversations, and structured output.
- Experience designing and implementing LLM output evaluation frameworks, including adversarial and security testing.
- Hands-on experience with RAG architectures, embedding models, vector stores, chunking strategies, and retrieval evaluation.
- Proficiency with major LLM APIs such as OpenAI, Google Gemini, Anthropic, or equivalent.
- Strong Python skills and comfort working in AWS environments.
- Familiarity with TTS and STT voice AI services.
- Ability to explain AI system behavior to creative, brand, and other non-technical collaborators.
- Familiarity with NVIDIA NeMo Guardrails, Guardrails AI, or equivalent is preferred.
- Conceptual understanding of fine-tuning and RLHF is preferred.
- Entertainment, gaming, or IP-driven product experience is preferred.
Benefits
- Health and wellness benefits.
- Time off to recharge.
- Financial well-being programs.
- Life and family support.
- Volunteer and community initiatives.
- Learning and development opportunities.
- Exclusive perks.
- Annual and long-term incentives may be available based on role, location, and eligibility.
Categories
