2 months ago
Seattle, WA, USA +2 moreMid Level
H1B sponsor
Base Salary
$180k - $225k/yr
Responsibilities
- Design and deploy production AI agents using large language models, reasoning, retrieval, memory, and tool use.
- Architect systems combining LLMs, traditional machine learning, structured knowledge, enterprise data, and deterministic software.
- Build retrieval pipelines, memory systems, knowledge representations, customer intelligence layers, and multi-agent workflows.
- Own experimentation and evaluation using benchmarks, online A/B experiments, golden datasets, regression suites, LLM-as-a-Judge, human evaluation, and ablation studies.
- Develop confidence estimation, reflection, continuous learning, guardrails, fallback strategies, tracing, monitoring, and evaluation pipelines.
- Deploy reliable, observable, safe, low-latency AI systems in enterprise cloud environments with infrastructure teams.
- Partner with customers to understand business and data challenges, prototype solutions, lead technical design, and evolve successful systems into scalable capabilities.
Requirements
- At least 4 years of software engineering, machine learning, or applied AI experience.
- Strong Python programming skills and experience building production AI systems using LLMs.
- Experience with modern AI tooling such as OpenAI, Claude, MCP, agent frameworks, vector databases, or retrieval systems.
- Strong understanding of machine learning fundamentals and modern language models.
- Experience designing or evaluating AI systems with quantitative metrics.
- Preferred experience includes production AI agents, autonomous systems, reasoning, retrieval, memory, planning, tool use, RAG, semantic search, knowledge graphs, fine-tuning, distillation, reinforcement learning, multimodal AI, or foundation models.
- Preferred software engineering experience includes distributed production systems, cloud platforms, Docker, Kubernetes, production observability, and enterprise software integration.
- Ability to work directly with enterprise customers and translate ambiguous business problems into technical architectures.
- Strong written and verbal communication skills; experience leading technical workshops, architecture reviews, or customer design sessions is preferred.
Benefits
- Comprehensive health, dental, and vision coverage.
- Retirement benefits, learning and development stipend, and generous paid time off.
- Potential commuter stipend and equity eligibility for eligible roles.
- Full-time position located in San Francisco, New York, or Seattle.
- Base salary range applies to the listed locations and is determined by location and qualifications.
Categories
About Scale AI
Scale AI builds data annotation services and AI development tools for enterprises and government agencies, sold as a platform and managed services. Its products include the Scale Generative AI Platform for building and evaluating agents and the Data Engine for collecting, curating, and labeling training data, including RLHF and model evaluation. Founded in 2016 and headquartered in San Francisco, the company is privately held and works across domains from computer vision to LLM applications.