Senior LLM Inference Performance & Evaluation Engineer

2 months ago

Singapore, SingaporeSenior

Responsibilities

Build benchmark pipelines for TTFT, inter-token latency, output throughput, request latency, concurrency, error rate, and GPU utilization.
Create model launch gates covering API compatibility, streaming, tool calling, reasoning output, multimodal behavior, and long-context cases.
Maintain synthetic, replayed, and customer-like workloads for steady-state and burst scenarios.
Compare model, runtime, and provider options and recommend routing, fallback, pricing, and capacity decisions.
Automate regression detection in CI/CD and staging for quality, latency, and cost changes.
Work with runtime engineers to identify bottlenecks and verify improvements, and with SRE to define SLOs and alert thresholds.

5+ years of experience in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
Experience with LLM serving metrics including TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU utilization.
Strong Python skills for benchmark and evaluation automation.
Go experience is a plus for integrating with platform services.
Familiarity with OpenAI and Anthropic APIs, vLLM, Dynamo, SGLang, Triton-style servers, and Kubernetes-based test environments.
Ability to design statistically meaningful tests and communicate tradeoffs involving model quality, latency, reliability, cost, and customer experience.
Experience building dashboards, reports, and release gates for engineering, product, and business stakeholders.

Culture valuing authenticity and diversity of thoughts and backgrounds.
Inclusive and respectful environment with open workspaces and startup spirit.
Opportunity to network with industry pioneers and enthusiasts at a fast-growing company.
Ability to contribute directly to the future of the digital asset industry.
Involvement in new projects and development of processes and systems.
Personal accountability, autonomy, fast growth, and learning opportunities.
Welfare benefits and developmental opportunities including training and mentoring.

201-500 employees