Member of Technical Staff (Model Behavior Architect)
Perplexity6 months ago
Base Salary
$180k - $270k/yr
Responsibilities
- Design, test, and optimize context strategies and system prompts for the answer engine.
- Build automated and semi-automated evaluation pipelines to measure model quality.
- Partner with research and engineering to validate model behavior during rollouts.
- Identify inconsistencies and failure modes in model outputs through research projects.
- Collaborate with design, product, and research teams to translate goals into model behavior requirements.
- Help engineers build intuition for prompt design and evaluation best practices.
- Stay updated on the latest alignment, evaluation, and prompting techniques.
Requirements
- Experience designing evaluations, benchmarks, or metrics for AI systems.
- Strong written and verbal communication skills for diverse stakeholders.
- Ability to manage multiple projects in a fast-paced environment.
- Strong experience with Perplexity or other frontier AI models in production.
- Demonstrated experience with Python for prototyping and building systems.
- 3+ years of experience working with LLMs in a product or research setting.