4 days ago
Bengaluru, IndiaMid Level / Senior
H1B Sponsor
Responsibilities
- Develop asynchronous inference systems for non-blocking client requests.
- Design intelligent request routing systems to balance load across model replicas.
- Implement zero-downtime model updates for seamless transitions between versions.
- Architect frameworks for multi-model orchestration in complex ML pipelines.
- Build observability tools for debugging distributed ML applications.
Requirements
- Strong understanding of operating systems, networking, concurrency, and distributed systems.
- Experience building and maintaining systems that serve real users at scale.
- High standards for code quality, simplicity, and testing coverage.
- Ownership mindset for code from design to deployment and incident response.
- Familiarity with distributed systems frameworks like gRPC and Ray is a plus.