8 months ago
Responsibilities
- Lead technical discovery to define customer use cases, workloads, infrastructure constraints, and model quality, latency, throughput, and cost targets.
- Optimize LLM inference using quantization, batching, prefix caching, KV-cache optimization, parallelism, and related techniques.
- Adapt models through prompt optimization, retrieval-augmented generation, and fine-tuning, and evaluate the impact of each approach.
- Troubleshoot model execution, GPU utilization, memory consumption, serving infrastructure, and application performance issues.
- Run customer workshops, present benchmark findings, explain technical tradeoffs, and guide deployment and optimization decisions.
- Build evaluation tools, benchmark harnesses, deployment configurations, and runbooks, and share recurring findings with product and engineering teams.
- Deploy AI solutions that meet agreed quality and performance targets while improving latency, throughput, GPU utilization, and cost.
Requirements
- At least 5 years of experience in software engineering, ML engineering, AI infrastructure, or a related hands-on technical role.
- Strong Python programming skills and experience with PyTorch, Hugging Face Transformers, or comparable tools.
- Hands-on experience deploying and optimizing production AI models, particularly large language models.
- Experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar technologies.
- Understanding of GPU memory constraints, numerical precision, batching, caching, and parallelism and their effects on inference performance.
- Experience building evaluations and benchmarks for model quality, latency, throughput, and resource efficiency.
- Experience developing APIs and integrating model endpoints with production applications and customer data systems.
- Familiarity with GPU-based cloud environments, containers, and monitoring tools used to operate model deployments.
- Ability to work directly with customers, communicate technical tradeoffs, and translate business requirements into measurable engineering targets.
- Strong ownership and troubleshooting skills, including the ability to turn ambiguous problems into tested solutions.
- Preferred experience with LoRA, QLoRA, speculative decoding, multi-GPU inference, distributed model serving, CUDA, NVIDIA Nsight, embedding models, rerankers, vision or multimodal models, retrieval quality, end-to-end RAG applications, open-source model-serving or evaluation projects, and enterprise AI implementations.
Benefits
- Medical, dental, and vision insurance fully paid by CoreWeave for US-based full-time employees.
- Company-paid life insurance, voluntary supplemental life insurance, short- and long-term disability insurance, Flexible Spending Account, and Health Savings Account.
- Tuition reimbursement and eligibility to participate in the Employee Stock Purchase Program.
- Mental wellness benefits through Spring Health and family-forming support through Carrot.
- Paid parental leave and flexible full-service childcare support through Kinside.
- 401(k) with an employer match and flexible paid time off.
- Catered lunch each day at office and data center locations.
- Casual work environment and a culture focused on innovative disruption.
- Benefits vary for roles outside the US and are shared during the hiring process.
About CoreWeave
CoreWeave provides a GPU-accelerated cloud for AI training and inference, VFX, and rendering, with bare-metal instances, Kubernetes orchestration, and managed services to scale workloads. It sells on-demand and reserved capacity to AI labs, startups, and enterprises, and offers SaaS tools and hands-on support for deployment. Founded in 2017 and headquartered in New York, it is publicly traded on Nasdaq under the ticker CRWV.