4 months ago
Base Salary
$201k - $264k/yr
Responsibilities
- Design and build scalable, fault-tolerant infrastructure systems across multiple cloud regions
- Own and evolve multi-cloud infrastructure, including Kubernetes orchestration, networking, and container management
- Lead observability, incident response, and operational excellence initiatives
- Architect and optimize distributed systems for reliability, including load balancing, quota management, and failover mechanisms
- Partner with Product Engineering and Security teams on infrastructure strategy and delivery
- Drive infrastructure-as-code practices with Terraform and Pulumi to enable reproducible, auditable deployments
- Design a model proxy architecture for millions of daily inference requests and seamless model integration
- Build distributed rate limiting and quota management systems using Redis-backed algorithms
- Architect multi-region deployments that satisfy global data residency requirements
- Develop observability infrastructure with SLA monitoring, burn rate alerts, and token attribution for cost tracking
- Lead the evolution of CI/CD pipelines while maintaining production stability
- Mentor engineers and raise the technical bar through code reviews, design reviews, and technical leadership
Requirements
- 10+ years of experience in infrastructure engineering or platform engineering in a production environment
- Extensive experience building and scaling complex, large-scale distributed systems
- Deep proficiency with cloud infrastructure platforms, with Azure preferred and GCP or AWS experience accepted
- Strong fluency with infrastructure-as-code tools such as Terraform, Pulumi, or CloudFormation
- Strong understanding of Kubernetes, container orchestration, networking, and cloud security at scale
- Experience with observability tools such as Datadog and Sentry and incident response practices using PagerDuty or Incident.io
- Strong programming skills in Python, Go, or similar languages
- Experience with AI/ML infrastructure or high-throughput inference systems is preferred
- Experience with distributed rate limiting, load balancing, or quota management systems is preferred
- Experience operating multi-tenant platforms with strict security and compliance requirements is preferred
- Track record of leading complex cross-functional projects and delivering measurable impact is preferred
Benefits
- In-person work model in New York City, New York
- Relocation assistance for new employees
Tech Stack
Categories
About Harvey
Harvey is domain-specific AI for legal and professional services. Adopted by Fortune 500 companies like AT&T, Verizon, Cox, Koch, KKR, Bridgewater, more than 100,000 lawyers across 2,400+ customers in 70 countries and over 75% of AmLaw 100 law firms rely on Harvey to advance legal expertise faster across contract analysis, due diligence, compliance, and litigation. Backed by Sequoia Capital, OpenAI, GV, Kleiner Perkins, Coatue and EQT, Harvey is the trusted partner in modernizing the legal industry.