ML Research Engineer, Performance Optimization
Lila Sciences1 day ago
Cambridge, MA, USA or San Francisco, CA, USAEntry Level / Mid Level
H1B sponsor
Base Salary
$148k - $240k/yr
Responsibilities
- Profile GPU workloads, develop kernels, support long-context handling, and optimize memory footprints.
- Optimize GPU cluster performance, including collective and MoE frameworks, communication/compute overlap, RDMA/IB/EFA usage, and P/D disaggregation.
- Manage mid-training-scale data across storage, streaming, tokenization, and processing systems.
- Optimize asynchronous reinforcement learning pipelines, including environment interfacing, long rollouts, token metadata, and weight synchronization.
- Create tested, accessible, reusable, and adaptable optimized configurations for research teams.
- Evaluate throughput and accuracy tradeoffs to establish optimal performance configurations.
- Maintain production libraries, clean APIs, integration tests, build pipelines, and automated quality gates.
Requirements
- BS/MS in Computer Science or a related quantitative field plus 4+ years of software engineering or machine learning engineering experience, or a relevant PhD plus 1+ years of work experience.
- Experience using and optimizing throughput for LLM inference libraries such as vLLM and training libraries such as Megatron.
- Experience running LLM inference or training at scale across multi-node GPU clusters.
- Familiarity with core reinforcement learning concepts.
- Strong software engineering skills, including clean APIs, well-tested code, and thoughtful abstractions.
- Hands-on experience supporting and maintaining production libraries used by engineers and researchers.
- Strong CI/CD experience, including integration test suites, build pipelines, and automated quality gates for multi-package ecosystems.
- Passion for agentic systems and awareness of current approaches to LLM tool use and autonomy.
- Preferred experience with CUDA, CuTe, CuTile, Triton, NCCL, NIXL, NVLink, IB, EFA, Ray, Flyte, Airflow, Docker, Modal, Ray actors, or Ray Serve.
Benefits
- Full-time U.S. employees receive medical, dental, and vision coverage.
- Employer-paid life and disability insurance, flexible time off, generous company-wide holidays, and paid parental leave are provided.
- Educational assistance, commuter benefits including bike-share memberships for office-based employees, and a company-subsidized lunch program are available to eligible U.S. employees.
- International employees receive a comprehensive benefits program tailored to their region.
Tech Stack
Apache AirflowDocker
Categories
About Lila Sciences
Lila Sciences builds an AI-driven research platform and autonomous lab systems for life science, chemistry, and materials R&D teams. Its products combine large AI models with robotic instruments to plan, run, and analyze experiments, integrating into customer-specific scientific workflows. The privately held company serves pharma, materials, and energy organizations and is venture-backed, including a Series A financing in 2025.