Responsibilities
- Design and build scalable Python backend services and pipelines for agent evaluation, observability, and quality measurement.
- Own evaluation platform components including golden datasets, LLM-as-judge pipelines, trajectory and correctness scoring, and regression detection.
- Build and operate agent observability infrastructure using Langfuse or equivalent OpenTelemetry-based tracing.
- Author and maintain Kubeflow Pipelines, including DAG authoring, artifact tracking, caching or memoization, and deployment on internal Kubernetes infrastructure.
- Drive initiatives end to end from design through production launch while partnering with product agent teams.
- Design and develop high-performance distributed applications and services with approximately 80–95% hands-on development and coding.
- Mentor and coach engineers on software engineering practices, platform architecture, and AI or agent evaluation concepts.
- Collaborate with data scientists, product managers, and engineering teams to define and measure agent quality.
- Experiment with customers, address ambiguous problems, and help advance AI-native development practices.
Requirements
- Bachelor's degree in Computer Science or equivalent practical experience.
- 10+ years of experience building and operating distributed backend systems in production.
- Strong proficiency in Python and solid understanding of APIs, data modeling, testing, and CI/CD.
- Hands-on experience with LLM or agent systems, including prompt engineering, embeddings, RAG, LLM-as-judge evaluation, or agent frameworks such as LangGraph.
- Experience with ML pipeline orchestration using Kubeflow Pipelines, Argo Workflows, or similar tools, including pipeline authoring, artifact or version tracking, and Kubernetes-based deployment.
- Familiarity with observability and tracing systems for LLM or distributed applications, such as Langfuse or OpenTelemetry.
- Experience with at least one major cloud provider: AWS, GCP, or Azure.
- Self-starter with strong problem-solving skills, a track record of shipping, and excellent verbal and written communication.
- Strong cross-functional partnership skills and comfort working with data scientists and product teams.
Benefits
- Competitive compensation package with pay-for-performance rewards.
- Position may be eligible for cash bonus, equity rewards, and benefits under applicable plans and programs.
Categories
About Intuit
Intuit is a global technology platform that helps our customers and communities overcome their most important financial challenges. Serving millions of customers worldwide with TurboTax, QuickBooks, Credit Karma and Mailchimp, we believe that everyone should have the opportunity to prosper and we work tirelessly to find new, innovative ways to deliver on this belief. We encourage conversations on this page and will not delete comments that follow our terms of use. In order to keep this a safe community, the below posts may be removed: Repeated posts of the same content, spam or posts from fake accounts or profiles, offensive language or material, threats to others in the community, posts deliberately aimed to have a negative effect on the community or conversations.