Software Engineer, Inference Deployment

Anthropic

21 days ago

New York, NY, USA +2 more

Senior

H1B Sponsor

Base Salary

$320k - $485k/yr

Responsibilities

Own deployment orchestration for validated inference builds across GPU, TPU, and Trainium fleets.
Improve capacity-aware deployment scheduling to maximize throughput against constrained resources.
Extend deployment observability with dashboards and tools for tracking code in production.
Drive down cycle time from code merge to production with optimized pipeline architectures.
Optimize fleet rollout strategies for large-scale deployments with minimal service disruption.
Evolve self-service model onboarding for continuous deployment without engineering involvement.
Collaborate with teams across the Inference organization to integrate deployment automation.

5+ years of experience building deployment infrastructure at scale.
Strong software engineering skills with experience in complex state machines and multi-stage pipelines.
Experience with resource-constrained deployment systems.
Proven track record of building automation that improves deployment velocity and reliability.
Proficiency with Kubernetes-based deployments and container orchestration.
Comfort working across the stack from backend services to web UIs.
Strong communication skills for collaboration with various teams.

KubernetesPythonRust