Senior Software Engineer, Machine Learning Infrastructure

about 1 month ago

H1B Sponsor

Base Salary

$176k - $220k/yr

Responsibilities

Build and operate shared infrastructure for production ML and AI, including data pipelines, feature stores, training, and model serving.
Develop and scale the LLM platform through provider integrations, orchestration, observability, and controls for cost, latency, and reliability.
Build LLM evaluation harnesses, benchmarks, and quality measurement pipelines.
Support post-training workflows including fine-tuning, reinforcement learning pipelines, and related data infrastructure.
Optimize inference infrastructure for open and fine-tuned models through GPU serving, batching, and autoscaling.
Partner with AI, data science, and product teams to productionize models and establish ML infrastructure best practices.
Improve the reliability, scalability, and developer experience of the ML platform.

At least 5 years of production software engineering experience using Python, Go, TypeScript, or similar languages.
Experience building and operating cloud infrastructure on AWS, GCP, or similar platforms.
Strong experience with Kubernetes, Docker, Terraform, CI/CD, and production service operations.
Hands-on experience building ML infrastructure such as model serving, training pipelines, feature stores, embeddings, or ML observability.
Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines.
Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization.
Strong systems design skills, sound engineering judgment, and the ability to work in ambiguous, fast-moving environments.
Extra credit for experience with Ray, Anyscale, KubeRay, Ray Serve, vLLM, Triton, PyTorch, GPU-backed inference and training, LLM evaluation frameworks, benchmarking systems, quality regression testing, Vertex AI, Bigtable, Redis, feature platforms, fine-tuning, RLHF, reinforcement learning, reward modeling, agentic systems, MCP integrations, tool use, memory systems, or voice AI applications.

Equity in a fast-growing company; 401(k) match, competitive compensation, and financial coaching; paid parental leave, fertility benefits, and parental coaching; medical, dental, vision, and mental health support; $500 wellness stipend; $2,000 learning stipend; remote and office support including internet, commuting, and free lunch/gym in the San Francisco office; flexible PTO, 15 holidays, and 2 flex days; team outings and referral bonuses.