2 hours ago
Base Salary
$160k - $240k/yr
Responsibilities
- Design and build scalable infrastructure for online and offline inference workloads.
- Integrate high-performance inference runtimes and serving frameworks including TensorRT, vLLM, ONNX, and Triton.
- Drive architecture and technical decisions across the inference platform while balancing latency, throughput, reliability, and cost.
- Partner with engineering teams to improve model deployment, observability, and production performance.
- Mentor junior engineers in system design, debugging, and performance optimization.
- Develop autoscaling, deployment pipeline, model-serving, structured sampling, prompt-caching, and serving-optimization capabilities.
Requirements
- 5+ years of professional software engineering experience.
- Experience designing, building, and operating production distributed systems.
- Strong systems intuition and experience debugging and optimizing performance-critical services.
- Ability to own problems end-to-end and ramp up quickly in unfamiliar technical areas.
- 4+ years of demonstrated experience with an object-oriented programming language.
- A degree in Computer Science or Electrical Engineering, or equivalent practical experience.
- Preferred: experience deploying and operating machine learning systems at scale.
- Preferred: experience with inference optimization techniques such as batching, caching, request scheduling, or memory-aware serving.
- Preferred: familiarity with PyTorch, CUDA, NCCL, NVLink, InfiniBand, MPI, Kubernetes, cloud-native infrastructure, load balancing, request routing, or traffic management systems.
Benefits
- Benefits may include merit increases, incentive compensation for exempt roles, paid holidays, paid time off, medical, dental, vision, short- and long-term disability benefits, 401(k) match, life insurance, and wellness programs.
- The posting states that benefits are not provided directly to contingent workers, contractors, or interns.
Tech Stack
Categories
About Bloomberg
Bloomberg is a global leader in business and financial information, delivering trusted data, news, and insights that bring transparency and efficiency, and fairness to markets. We help connect influential communities across the global financial ecosystem via reliable technology solutions that enable our customers to make more informed decisions and foster better collaboration. We challenge the status quo through constant innovation. We collaborate broadly because we know that other perspectives matter. We put our customers first, as a guiding beacon. And we believe doing the right thing – by our people, our clients, and our communities – is the best thing for our business.