14 days ago
Base Salary
$320k - $485k/yr
Responsibilities
- Bring up inference for new model architectures and launch them across cloud platforms in sync with Anthropic’s first-party platform.
- Integrate new inference features such as structured sampling and prompt caching into cloud platforms and own their production delivery.
- Identify and fix cross-platform inference gaps involving configuration drift, observability, deployment patterns, and difficult provider-specific bugs.
- Design, build, and own CI/CD infrastructure for inference servers and load balancers across cloud platforms.
- Develop shadow-traffic validation, throughput and latency baselines, and correctness checks to detect regressions before production.
- Reduce merge-to-production time by making validation faster, more parallel, reliable, and cost-effective on constrained accelerator pools.
- Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation.
Requirements
- Significant software engineering experience with a strong background in high-performance, large-scale distributed systems serving millions of users.
- A track record of building automation or test infrastructure that measurably improved release velocity or reliability.
- Experience building or operating services on at least one major cloud platform: AWS, GCP, or Azure.
- Exposure to Kubernetes, infrastructure as code, or container orchestration.
- Strong interest in LLM serving; prior inference or machine learning experience is not required.
- Ability to collaborate with internal teams and external partners, learn new technologies and hardware platforms quickly, and own problems end to end.
- Preferred experience with LLM inference optimization, batching, caching strategies, capacity-constrained scheduling, or shared-resource test infrastructure.
- Preferred understanding of multi-region deployments, request routing, load balancing, global traffic management, and cross-provider infrastructure scaling.
- Preferred experience navigating networking, security, privacy, and managed-service differences across cloud providers.
- Proficiency in Python or Rust is preferred.
- Bachelor’s degree in a relevant field, or an equivalent combination of education, training, and/or experience.
Benefits
- Annual compensation is listed as $320,000–$485,000 USD.
- Hybrid policy currently expects staff to work from an office at least 25% of the time, with some roles requiring more office time.
- Anthropic sponsors visas where possible and provides immigration-lawyer support for sponsored candidates.
- Benefits include competitive benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.
Categories
About Anthropic
We're an AI research company that builds reliable, interpretable, and steerable AI systems. Our first product is Claude, an AI assistant for tasks at any scale. Our research interests span multiple areas including natural language, human feedback, scaling laws, reinforcement learning, code generation, and interpretability.