9 hours ago
Base Salary
$152k - $261k/yr
Responsibilities
- Architect and build an AI Inference Gateway platform for large-scale machine learning and generative AI workloads.
- Design request routing, model endpoint abstraction, traffic shaping, load balancing, failover, caching, and policy enforcement mechanisms.
- Develop infrastructure components in Go, Java, or Python across cloud and on-premise environments.
- Build multi-tenant capabilities including authentication, authorization, quotas, cost attribution, rate limiting, and governance controls.
- Partner with ML Platform, Model Serving, Data, and Product Engineering teams to operate AI workloads.
- Lead architecture and design reviews, establish engineering standards, and mentor senior engineers.
- Optimize inference latency, throughput, resiliency, and infrastructure efficiency.
- Define observability standards and operate business-critical AI services using metrics, tracing, logging, and SLO-based operations.
- Investigate production issues, conduct root-cause analysis, and implement long-term architectural improvements.
Requirements
- 6+ years of professional software development experience.
- 4+ years of experience designing and operating large-scale distributed systems in production.
- Strong programming expertise in one or more of Go, Java, or Python.
- Experience building highly available, mission-critical platform or infrastructure services.
- Experience designing API platforms, service gateways, service meshes, or large-scale networking infrastructure.
- Deep understanding of distributed systems, microservices architecture, and cloud-native technologies.
- Production experience with Kubernetes and containerized workloads.
- Experience operating services on AWS, Azure, or GCP.
- Strong understanding of observability, reliability engineering, capacity planning, and production operations.
- Preferred experience with AI/ML inference platforms, LLM gateways, model serving infrastructure, or GPU-accelerated workloads.
- Preferred expertise with Kubernetes networking technologies, inference-serving frameworks, high-performance networking, traffic management, GenAI and LLM ecosystems, distributed data systems, and asynchronous programming.
Benefits
- Medical, dental, vision, life, AD&D, disability, FSA, HSA, EAP, and 401(k) benefits.
- 18–21 days of paid time off annually based on tenure and 12 public holidays.
- Paid parental leave and pre-tax commuter benefits.
- Free electric car charging station.
- Base pay ranges from $152,000 to $261,000 per year depending on geographic market and job-related factors.
Tech Stack
AmbassadorApache CassandraApache KafkaArgo CDAWSAzureGoGoogle Cloud PlatformgRPCIstioJavaKubernetesMongoDBPythonRedis
Categories
About Coupang
Coupang builds and operates a South Korea–focused e-commerce marketplace with an end-to-end logistics network (Rocket Delivery), plus food delivery, video streaming, and fintech under brands such as Coupang, Eats, and Play. Revenue comes from first-party retail, third-party marketplace services, advertising, and memberships (Rocket WOW). Founded in 2010, the company is headquartered in Seattle and is publicly listed on the NYSE (CPNG).
