2 days ago
San Francisco, CA, USA or New York, NY, USAMid Level
H1B sponsor
Base Salary
$200k - $400k/yr
Responsibilities
- Build and operate the Model APIs layer, including request handling, authentication, rate limiting, routing, high availability, and low latency.
- Own real-time usage metering and billing infrastructure that supports evolving pricing and packaging models.
- Build rate-limiting and capacity systems that balance customer throughput with fleet utilization at scale.
- Ship API-layer products such as model routing and developer usability features.
- Deliver usage visibility, self-service controls, and predictable API behavior for developers.
- Instrument the platform edge with observability and alerting to identify issues proactively.
- Partner with Model APIs, inference, and product teams to deliver capabilities safely and quickly.
- Own projects end to end from design through deployment, on-call, and customer-feedback iteration.
Requirements
- Bachelor’s, master’s, or Ph.D. in computer science, engineering, or a related field, or equivalent practical experience.
- 3+ years of experience building and operating distributed systems, backend infrastructure, or large-scale APIs.
- Demonstrated ownership of low-latency, reliable backend services involving rate limiting, authentication, quotas, metering, or migrations.
- Experience with profiling, tracing, capacity planning, and SLO management.
- Ability to debug performance and reliability issues across application, runtime, and infrastructure layers.
- Strong developer-experience orientation and ability to consider how systems are used.
- Willingness to learn new languages, frameworks, and systems, with interest in inference engineering.
- Excellent written communication and cross-functional collaboration skills, including design documentation.
- Deep Kubernetes experience, including operators and custom resources, is preferred.
- Familiarity with service meshes, API gateways, distributed scheduling, autoscaling, service orchestration, observability tooling, CI/CD systems, release automation, or developer-facing infrastructure is preferred.
- Prior model-serving, inference, ML, LLM, or open-source infrastructure experience is a plus but is not required.
Benefits
- Competitive compensation including meaningful equity.
- 100% company-paid medical, dental, and vision insurance for U.S. employees and dependents.
- Flexible PTO and a company-wide Winter Break from Christmas Eve through New Year’s Day.
- Paid parental leave.
- Fertility and family-building stipend through Carrot.
- Company-facilitated 401(k) for U.S. employees.
- Exposure to a variety of ML startups and associated learning and networking opportunities.
Tech Stack
Categories
About Baseten
Baseten builds an AI inference platform that provides tooling, infrastructure, and hardware to deploy, scale, and serve machine-learning models in production. The company sells managed model serving and developer tooling to software teams at AI product companies, with customers including Notion, Abridge, Writer, and Cursor. Privately held and headquartered in San Francisco, it focuses on high-availability, globally distributed inference for production workloads.
