2 months ago
Remote, United StatesSenior
Base Salary
$170k - $210k/yr
Responsibilities
- Lead the design and build of the AI inference platform, including architecture patterns, deployment standards, and operational practices.
- Own end-to-end AI model-serving infrastructure across on-premises and data center environments.
- Build and maintain fault-tolerant, high-performance systems optimized for latency, reliability, scalability, and cost efficiency.
- Integrate AI inference data and configuration with power optimization algorithms in collaboration with algorithms engineers.
- Optimize GPU utilization and inference performance across NVIDIA-based hardware fleets.
- Establish MLOps practices, including CI/CD pipelines for model deployment, monitoring, and rollback across environments.
- Contribute to infrastructure roadmap, tooling selection, and build-versus-buy decisions.
- Collaborate cross-functionally with product, engineering, and data science teams.
- Travel up to 10% of the time.
Requirements
- At least 5 years of software engineering experience focused on AI infrastructure, backend systems, or distributed systems.
- Hands-on experience with AI model-serving frameworks such as vLLM, SGLang, Triton, TensorRT, TorchServe, or similar.
- Experience with Kubernetes, Docker, container orchestration, and cluster management.
- Experience deploying and operating infrastructure across data center and on-premises environments.
- Strong understanding of GPU workloads and the differences between AI inference and training.
- Proficiency in Python.
- C++, CUDA, and Go or Rust experience is a plus.
- Dynamo experience is a plus.
- Experience with edge AI deployments or constrained compute environments is preferred.
- Familiarity with Terraform and Helm is preferred.
- Experience with Datadog, Prometheus, or Grafana is preferred.
- Background in energy, utilities, or industrial IoT is preferred.
- Contributions to open-source ML infrastructure projects are preferred.
- Strong communication skills and comfort working cross-functionally in a lean, fast-moving environment.
Benefits
- Fully remote work from anywhere in the United States.
- Periodic travel for company retreats and key on-site engagements, with travel expected up to 10% of the time.
- Flexible work environment with flexible paid time off.
- Health, dental, and vision benefits.
- Employer-match 401k.
- Stock options.
- Mentorship and growth opportunities.
- Collaborative, diverse, and inclusive workplace.
Categories
About Utilidata
Utilidata builds Karman, an AI and edge-computing platform that orchestrates power for AI data centers and provides real-time visibility and control on the electric distribution grid. It sells software and embedded modules (based on a custom NVIDIA Jetson Orin Nano) to data center operators and electric utilities. Founded in 2012 and privately held, the company is headquartered in Ann Arbor, Michigan, and has deployed Karman in smart meters and other grid devices.