Senior AI Engineer
DV Trading1 month ago
Hong Kong, Hong Kong +4 moreSenior
Base Salary
$200k - $300k/yr
Responsibilities
- Build and operate a model gateway that routes inference across open and closed models while tracking cost, latency, and quality.
- Design and run distillation pipelines that use frontier-model outputs to generate training data for task-specific open models.
- Fine-tune and evaluate open-weight models such as Llama, Qwen, and Mistral for DV-specific tasks.
- Deploy and maintain on-premise inference infrastructure using vLLM, TGI, or equivalent tools on Kubernetes.
- Build model evaluation frameworks for quality, cost, latency, and regression testing.
- Define model-selection criteria and tooling for determining when open models are production-ready versus when closed APIs should be used.
- Partner with the agent engineering team to ensure the model layer supports agent workloads.
- Manage production GPU infrastructure, including provisioning, scheduling, and utilization monitoring.
Requirements
- At least 5 years of software engineering experience and strong Python skills.
- Production experience fine-tuning or distilling open-weight models, beyond building inference API wrappers.
- Experience serving LLMs on-premise using vLLM, TGI, Triton, or equivalent technology.
- Experience managing production GPU infrastructure, including provisioning, scheduling, and utilization monitoring.
- Experience with production model evaluation and regression testing.
- Experience with Kubernetes and GPU workload management.
- Strong understanding of tradeoffs among open and closed models involving cost, quality, latency, and data sensitivity.
- Preferred experience with quantization, PEFT/LoRA, or other efficient training techniques.
- Preferred experience designing model gateways or inference proxies with routing, fallback, and rate limiting.
- Preferred experience in financial services or other regulated or sensitive-data environments.
- Preferred familiarity with the open model ecosystem, including Hugging Face, model cards, and licensing.
Benefits
- Discretionary bonus eligibility.
- Medical, dental, and vision insurance.
- HSA, FSA, and Dependent Care options.
- Employer-paid group term life and AD&D insurance.
- Voluntary LTD, life, and AD&D insurance.
- Flexible vacation policy.
- Retirement plan with employer match.
Tech Stack
Categories