Member of Technical Staff (Answer Quality & Evals)
Perplexity7 months ago
Palo Alto, CA, USA or San Francisco, CA, USAMid Level / Senior
H1B sponsor
Base Salary
$200k - $350k/yr
Responsibilities
- Build shared infrastructure for reliable evaluations, result analysis, and product and model decision-making.
- Develop platforms for replaying and analyzing agent traces to reproduce production behavior and diagnose failures.
- Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data.
- Partner with data scientists, engineers, and product teams to convert answer-quality problems into evaluations, analyses, and product improvements.
- Own ambiguous technical projects from initial design through production operation in a small, high-impact team.
Requirements
- At least 4 years of software, data, or machine learning engineering experience shipping and operating production systems.
- Strong proficiency in Python and SQL, with solid fundamentals in system design, data modeling, and distributed systems.
- Experience building big-data systems involving distributed compute, large-scale storage, and high-volume pipelines.
- Demonstrated ownership of ambiguous technical projects from design through production operation.
- Ability to work effectively with data scientists, engineers, and product partners.
- Preferred: experience with evaluation, experimentation, observability, or machine learning infrastructure.
- Preferred: familiarity with LLM and agent systems, including tool use, execution traces, replay, and simulation.
- Preferred: experience with large-scale data processing platforms such as Databricks, Snowflake, or ClickHouse.
Categories
AI ApplicationsData Engineering
About Perplexity
Perplexity builds an AI-powered answer engine for web and mobile and an enterprise product, Perplexity Computer, for AI-driven workflows across tools and apps. The company monetizes through individual subscriptions (Perplexity Pro) and enterprise plans. Founded in 2022 and headquartered in San Francisco, it operates as a privately held company.