3 months ago
Seattle, WA, USA +2 moreSenior / Staff+
H1B sponsor
Base Salary
$320k - $485k/yr
Responsibilities
- Design, build, and maintain distributed systems serving Claude to millions of users worldwide.
- Develop intelligent request routing, load balancing, and traffic management across thousands of accelerators.
- Maximize fleet compute efficiency through autoscaling and orchestration of production, research, and experimental workloads.
- Build and operate production-grade pipelines for deploying new models to users.
- Provide high-performance inference infrastructure for model research and development.
- Integrate new AI accelerator platforms and support inference for new model architectures.
- Use observability data and real-world production workloads to tune and improve performance.
- Manage multi-region deployments and geographic routing for global customers.
- Contribute to inference features such as structured sampling and prompt caching.
Requirements
- Significant software engineering experience, particularly with distributed systems.
- Experience with high-performance, large-scale distributed systems is preferred.
- Experience implementing and deploying machine learning systems at scale is preferred.
- Experience with load balancing, request routing, or traffic management systems is preferred.
- Familiarity with LLM inference optimization, batching, and caching strategies is preferred.
- Experience with Kubernetes and cloud infrastructure such as AWS, GCP, or Azure is preferred.
- Proficiency in Python or Rust is preferred.
- A bachelor’s degree or an equivalent combination of education, training, and/or experience is required.
- The required field of study must be relevant to the role as demonstrated through coursework, training, or professional experience.
- Candidates should be flexible, impact-oriented, collaborative, willing to learn machine learning systems and infrastructure, and attentive to the societal impacts of their work.
Benefits
- Competitive benefits, generous vacation and parental leave, flexible working hours, optional equity donation matching, and office collaboration space.
- Hybrid policy requiring staff to work from an office at least 25% of the time.
- Visa sponsorship may be available, with immigration-lawyer support.
- Applications are reviewed on a rolling basis with no stated deadline.
Categories
About Anthropic
We're an AI research company that builds reliable, interpretable, and steerable AI systems. Our first product is Claude, an AI assistant for tasks at any scale. Our research interests span multiple areas including natural language, human feedback, scaling laws, reinforcement learning, code generation, and interpretability.