Staff Software Engineer - GenAI inference
Databricks11 months ago
Base Salary
$191k - $233k/yr
Responsibilities
- Own the architecture, design, and implementation of the GenAI inference engine and collaborate on an LLM-optimized model-serving stack.
- Partner with researchers to integrate new model architectures and features such as sparsity, activation compression, and mixture-of-experts.
- Lead optimization of latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
- Define standards and guide instrumentation, profiling, and tracing tooling to identify bottlenecks.
- Architect routing, batching, scheduling, memory management, and dynamic-loading mechanisms for inference workloads.
- Ensure reliability, reproducibility, and fault tolerance through A/B launches, rollback, and model versioning.
- Integrate with federated and distributed inference infrastructure across nodes, including load balancing and communication management.
- Collaborate with platform engineering, cloud infrastructure, and security/compliance teams.
- Represent the team through benchmarks, whitepapers, and open-source contributions.
Requirements
- BS, MS, or PhD in Computer Science or a related field, or equivalent experience.
- 6+ years or equivalent experience in performance-critical software systems.
- Strong ownership of complex system components and end-to-end architectural decision-making.
- Deep understanding of ML inference internals, including attention, MLPs, recurrent modules, quantization, and sparse operations.
- Hands-on CUDA, GPU programming, and experience with cuBLAS, cuDNN, and NCCL.
- Strong distributed-systems design background, including RPC frameworks, queuing, RPC batching, sharding, and memory partitioning.
- Experience identifying and resolving performance bottlenecks across kernels, memory, networking, and scheduling layers.
- Experience building instrumentation, tracing, and profiling tools for ML models.
- Ability to lead through influence and collaborate closely with ML researchers to productionize new model ideas.
- Strong communication, leadership, proactive ownership, and collaboration skills.
- Published research or open-source contributions in ML systems, inference optimization, or model serving are a bonus.
Tech Stack
Apache SparkMLflow
Categories
About Databricks
Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and over 60% of the Fortune 500 — rely on Databricks to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified Data Intelligence Platform that includes Agent Bricks, Lakeflow, Lakehouse, Lakebase and Unity Catalog. --- Databricks applicants Please apply through our official Careers page at databricks.com/company/careers. All official communication from Databricks will come from email addresses ending with @databricks.com or @goodtime.io (our meeting tool).