2 months ago
Santa Clara, CA, USASenior
Base Salary
$180k - $400k/yr
Responsibilities
- Own and evolve the core platform infrastructure, including API serving, state management, policy enforcement, and agentic workflow execution.
- Design and operate high-throughput, low-latency distributed services for model API products, including request routing, load management, rate limiting, and multi-tenant isolation.
- Build and maintain ETL/ELT pipelines for API logs, usage analytics, and billing.
- Develop production-grade internal SDKs and libraries with clean APIs, strong type safety, and clear contracts.
- Architect context and memory systems with low-latency retrieval, caching, vector-store integration, and retrieval pipelines.
- Instrument end-to-end observability with SLIs, SLOs, structured logging, and tracing, and drive platform reliability improvements.
- Collaborate with ML and product teams to integrate model serving, voice runtime, and tooling infrastructure.
Requirements
- 3+ years of experience building and operating backend systems at scale, including production services depended on by other teams.
- Strong distributed-systems fundamentals, including concurrency, fault tolerance, consistency tradeoffs, and capacity planning.
- Hands-on experience with data-pipeline infrastructure such as Kafka, Kinesis, Spark, Flink, or Airflow for log processing, analytics, or ETL workloads.
- Experience designing APIs and frameworks adopted by other engineering teams.
- Proficiency in at least one systems language such as Go, Rust, Java, or C++, or in Python in a performance-sensitive context.
- Comfort with cloud infrastructure, containerized deployments, CI/CD, and production on-call.
- Experience with LLM serving, agentic orchestration patterns such as ReAct or planner-executor, or RAG pipelines is a bonus.
- Familiarity with MCP, A2A, LangChain, or LlamaIndex is a bonus.
- Experience with real-time media systems or high-stakes platform services such as payments, identity, or core data is a bonus.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkAWSC++GoGoogle Cloud PlatformJavaKubernetesPythonReactRust
