3 months ago
Base Salary
$132k - $198k/yr
Responsibilities
- Architect and develop production-grade LLM-powered applications for internal automation, knowledge retrieval, and intelligent workflow assistance.
- Build reliable API integrations between AI models and HRIS, ticketing, CRM, and knowledge-base systems.
- Develop full-stack AI features spanning backend model orchestration, API layers, and frontend interfaces.
- Manage CI/CD pipelines, model evaluation frameworks, A/B testing infrastructure, and safe rollout strategies for AI systems.
- Design and implement RAG and retrieval systems grounded in Klaviyo’s internal knowledge.
- Lead model benchmarking, evaluation, fine-tuning, prompt engineering, and parameter-efficient fine-tuning.
- Define AI engineering standards, testing frameworks, and architectural patterns.
- Build observability and monitoring for latency, output quality, hallucination detection, costs, and production alerts.
- Collaborate with Product, Data Science, ML Engineering, and PM stakeholders to move AI features from experimentation into production.
- Apply security, privacy, governance, and compliance practices to AI integrations and sensitive data handling.
Requirements
- At least 3 years of software engineering experience, including at least 2 years building AI, ML, or LLM-powered systems in production.
- Deep expertise with large language models such as OpenAI, Anthropic, Mistral, or Llama and experience integrating them into production applications via API.
- Strong full-stack skills with Python, FastAPI, Flask, or Django, plus TypeScript or JavaScript and modern frontend frameworks.
- Experience designing and building CI/CD pipelines using tools such as GitHub Actions, Jenkins, or CircleCI.
- Experience with RAG systems, vector databases such as Pinecone, Weaviate, or pgvector, and embedding models.
- Proficiency with AWS, GCP, or Azure; Docker and Kubernetes; and infrastructure-as-code tools such as Terraform or Pulumi.
- Experience with prompt engineering, model evaluation frameworks, and LLM observability tools such as LangSmith or Weights and Biases.
Benefits
- Comprehensive health, welfare, and wellbeing benefits based on eligibility.
- The role may require up to 10% travel for onboarding, client or partner work, team meetings, and industry events.
- Travel is coordinated in advance.
Tech Stack
AWSAzureCircleCIDjangoDockerFastAPIFlaskGitHub ActionsGoogle Cloud PlatformJavaScriptJenkinsKubernetesPythonTerraformTypeScript
Categories
About Klaviyo
Klaviyo builds a subscription SaaS platform that lets e-commerce and consumer brands use first-party data to run targeted email, SMS, and push campaigns, with built-in automation and analytics. The product also includes B2C CRM features for segmentation, A/B testing, and customer lifecycle workflows. Founded in 2012 and headquartered in Boston, it is a public company (NYSE: KVYO) used by brands such as Mattel and Glossier.