3 hours ago
Base Salary
$350k - $850k/yr
Responsibilities
- Build and optimize Anthropic’s inference engine across accelerator and cloud platforms.
- Improve inference throughput, cost, reliability, and latency at Anthropic scale.
- Manage batching, model placement across chips, memory for weights and activations, forward-pass coordination, and model state across requests.
- Analyze FLOPs, HBM, PCIe, RDMA, network links, and other performance constraints.
- Profile systems, model potential improvements, implement changes, deploy them, and measure results.
- Improve device utilization through caching, reuse, observability, and performance optimization.
- Help maintain model quality, reproducibility, robustness, and safety across platforms and over time.
- Collaborate with safeguards and safety teams and participate in pair programming.
Requirements
- Bachelor’s degree or an equivalent combination of education, training, and/or experience in a relevant field.
- Working mental model of LLM inference, including prefill and decode behavior across accelerator compute, memory, interconnect, and host systems.
- Evidence of quickly learning unfamiliar deep systems and shipping consequential changes.
- Strong systems programming skills in Rust, C++, or a similar language, with attention to code quality and testing.
- Analytical approach to performance measurement, profiling, hypothesis formation, experimentation, and validation.
- Preferred: experience with an LLM serving engine and its abstractions.
- Preferred: GPU or accelerator programming experience.
- Preferred: OS internals, transformer language modeling, or experience building allocators, caches, schedulers, or high-bandwidth transport.
- Preferred: Rust fluency and experience with determinism, replay, and property-based tests.
- Collaborative, receptive to feedback, willing to work beyond a narrow job description, and comfortable with pair programming.
Benefits
- Annual salary range of $350,000–$850,000 USD.
- Visa sponsorship is offered with reasonable efforts made after an offer, subject to role and candidate circumstances.
- Staff are currently expected to work from an office at least 25% of the time, with some roles requiring more office time.
- Competitive benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.
- Anthropic is headquartered in San Francisco and operates as a public benefit corporation.
Categories
About Anthropic
Anthropic builds large language models and the Claude AI assistant for developers and enterprises, offered via API access and enterprise plans. Founded in 2021 and headquartered in San Francisco, it distributes Claude through its own platform and via partners such as Amazon Bedrock and Google Cloud’s Vertex AI. Its work emphasizes model reliability, interpretability, and practical tooling for tasks like coding assistance, analysis, and customer support automation.