Software Engineer, Inference
Anthropic
5 months ago
New York, NY, USA +2 more
Mid Level / Senior
H1B Sponsor
Base Salary
$300k - $485k/yr
Responsibilities
- Build and maintain systems for serving AI models to users.
- Maximize compute efficiency to support customer growth.
- Enable high-performance inference infrastructure for research.
- Design intelligent routing algorithms for request distribution.
- Autoscale compute fleets to match supply with demand.
- Build production-grade deployment pipelines for new models.
- Integrate new AI accelerator platforms.
- Analyze observability data to optimize performance.
Requirements
- Significant software engineering experience, especially with distributed systems.
- Results-oriented with a focus on flexibility and impact.
- Willingness to take on tasks outside of the job description.
- Enjoy pair programming and collaborative work.
- Interest in learning about machine learning systems and infrastructure.
- Experience with high-performance, large-scale distributed systems is a plus.
- Familiarity with Kubernetes and cloud infrastructure (AWS, GCP) is preferred.
- Proficiency in Python or Rust is advantageous.
Benefits
- Competitive compensation and benefits package.
- Optional equity donation matching.
- Generous vacation and parental leave.
- Flexible working hours.
- Collaborative office space.
Tech Stack
AWSGoogle Cloud PlatformKubernetesPythonRust
Categories
AI & MLBackendData Engineering