GoodData.AI

Sr. Inference Engineer (LLM Serving)

GoodData.AI
Apply
2 days ago
Prague, CzechiaSenior

Responsibilities

  • Deploy, operate, and upgrade open-weight LLMs and smaller task models on company GPUs across production regions.
  • Optimize inference performance and cost through batching, KV and prefix caching, quantization, parallelism, and quantitative latency and throughput tracking.
  • Evaluate new open-weight models on real analytics workloads and recommend production candidates.
  • Expand the inference gateway with intent and complexity-based routing, semantic caching, and weak-answer validation and escalation.
  • Adapt models for analytics use cases using LoRA adapters and fine-tuning.
  • Build expertise across the company for production model serving and inference systems.

Requirements

  • 5+ years of software engineering experience, including 2+ years running ML or LLM models in production.
  • Hands-on production experience with at least one of vLLM, SGLang, or TensorRT-LLM.
  • Working knowledge of inference bottlenecks including KV cache, attention, memory bandwidth, batching, and parallelism, with the ability to reason quantitatively about latency, throughput, and cost.
  • Experience with Docker, Kubernetes, and GPU instances in a public cloud.
  • A disciplined approach to benchmarking and model evaluation.
  • Clear communication in English with engineering and product stakeholders.
  • Preferred experience includes speculative decoding, multi-LoRA serving, PEFT fine-tuning, contributions to vLLM, SGLang, or PyTorch, CUDA or Triton kernels, NVIDIA GPU infrastructure, LLM routing, guardrails, embeddings, rerankers, Go or Rust, Terraform, Helm, OpenTelemetry, Langfuse, Prometheus, Grafana, analytics, semantic layers, or text-to-SQL.

Benefits

  • Hybrid work arrangement.
  • Opportunity to build and operate private, governed AI inference infrastructure using company-owned GPUs.
  • Work on an AI-native data intelligence platform serving enterprise customers.
GoodData.AI

About GoodData.AI

201-500 employees

GoodData.AI builds a cloud-based, AI‑native analytics and decision intelligence platform for enterprises and software vendors to build, embed, and govern analytics and AI agents. It provides composable analytics with a governed semantic layer, SDKs, and APIs for multi-tenant embedded deployments in the cloud or on-premises, sold via SaaS and professional services. Founded in 2007 and headquartered in San Francisco, the company is privately held.

Contact me