Senior AI Engineer
Procore Technologies1 day ago
Base Salary
$141k - $194k/yr
Responsibilities
- Improve online evaluations, including LLM judge accuracy, calibration, and production infrastructure.
- Build and maintain production trace infrastructure using Langfuse and Databricks.
- Build and maintain goldset data pipelines, including annotation and quality checks.
- Create agentic loops that identify failures in production traces and turn them into evaluation cases and agent improvements.
- Improve the runtime, cost, and reliability of online and offline evaluation pipelines.
- Contribute code and architecture, review designs, and mentor engineers.
Requirements
- 5+ years of experience in software, AI, or ML engineering.
- Strong proficiency in Python; TypeScript experience is a plus.
- Experience evaluating LLM systems through LLM-as-judge methods, rubric design, goldset curation, or judge-agreement measurement.
- Experience with LLM observability or evaluation tools such as Langfuse, Braintrust, or similar.
- Experience building data pipelines, ETL processes, or streaming data systems.
- Experience with analytics platforms and data warehouses such as Databricks or Snowflake.
- Ability to communicate evaluation results clearly and influence engineering teams.
- Degree in computer science or a related technical field, or equivalent practical experience.
Benefits
- Hybrid position based in the Austin, Texas office.
- Base pay range of $140,960.00-$193,820.00 annually, with potential eligibility for equity compensation and/or bonus incentive compensation.
Tech Stack
Categories
About Procore Technologies
Procore Technologies builds a cloud platform for construction project management used by general contractors, specialty contractors, and owners to manage bidding, scheduling, documents, financials, and field safety via web and mobile apps. It sells its software by subscription and integrates with common accounting and ERP systems. Founded in 2002 and headquartered in Carpinteria, California, Procore is a public company (NYSE: PCOR) whose software supports projects worldwide.