Machine Learning Engineer- Inference Optimization | Experienced Hire
Susquehanna International Group6 months ago
Responsibilities
- Design, build, and optimize low-latency inference systems for production machine learning workloads.
- Profile inference pipelines across model execution, runtime configuration, batching, memory movement, serialization, networking, and I/O.
- Evaluate, integrate, and tune inference runtime systems.
- Improve latency, throughput, and GPU utilization for production inference workloads.
- Build benchmarking and profiling tools for comparing model variants, hardware targets, runtime configurations, and deployment strategies.
- Debug performance issues involving GPU memory, compute saturation, kernel behavior, CPU/GPU coordination, data movement, and serving-layer overhead.
- Collaborate on model and system design choices and, when necessary, custom operators, kernel optimization, and hardware-specific performance work.
Requirements
- Experience deploying, optimizing, or operating machine learning inference workloads in production or production-like environments.
- Programming experience in Python, Java, C#, or similar, plus at least one systems language such as C, C++, Rust, or Go.
- Solid understanding of modern machine learning frameworks such as PyTorch, including model execution, export, tracing, compilation, and performance profiling.
- Ability to reason about latency, throughput, batching, memory use, GPU utilization, and reliability under real workloads.
- Experience optimizing inference for latency-sensitive or high-throughput applications is preferred.
- Experience with quantization, pruning, distillation, operator fusion, graph lowering, custom operators, or model compilation is preferred.
- Exposure to CUDA, Triton language, ROCm, PTX, CuTe, CUTLASS, FlashInfer, or similar low-level GPU programming tools is preferred.
- Experience running inference workloads on Kubernetes or GPU clusters, including scheduling, autoscaling, observability, and resource management, is preferred.
- A background in mathematics, physics, computer science, engineering, statistics, quantitative finance, or another technical field is preferred.
- Demonstrated ability to improve real-world inference performance beyond a baseline framework implementation is preferred.
Categories
About Susquehanna International Group
Susquehanna International Group (SIG) is a privately held quantitative trading firm and global market maker that uses proprietary technology and research to trade equities, options, ETFs, fixed income, and other derivatives. It earns revenue from proprietary trading and providing market liquidity, and also operates institutional sales & research and venture capital/private equity businesses. Founded in 1987 and headquartered in Bala Cynwyd, Pennsylvania, SIG has offices across North America, Europe, and Asia.