AI Platform Engineer
New York University16 hours ago
New York, NY, USAMid Level
Base Salary
$170k - $190k/yr
Responsibilities
- Build, operate, and continuously improve shared infrastructure and platform services supporting NYU’s enterprise AI capabilities.
- Develop and maintain cloud infrastructure, model gateways and routing, APIs, IAM, automated deployment pipelines, observability, usage and cost telemetry, and platform reliability capabilities.
- Translate AI architecture into production-ready infrastructure and reusable platform services.
- Configure and manage AI gateways and LLM proxy systems, including model routing, streaming responses, rate limiting, and operational controls.
- Build and support infrastructure-as-code, container orchestration, CI/CD pipelines, authentication, secrets management, and cloud network security.
- Set up and operate metrics, logging, distributed tracing, alerting, and reliability capabilities for AI services.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related quantitative field, or an equivalent combination of education and experience.
- At least 3 years of progressive experience in platform engineering, cloud engineering, DevOps/SRE, software engineering, or related infrastructure engineering roles.
- Direct experience operating production cloud services at scale.
- Deep experience with AWS cloud services and cloud-native architecture; AWS Bedrock experience is highly desirable.
- Proficiency with Terraform, AWS CloudFormation, or equivalent infrastructure-as-code tools.
- Hands-on experience with Docker and Kubernetes/EKS or AWS ECS.
- Hands-on experience building and supporting CI/CD pipelines using tools such as GitHub Actions, GitLab CI, or Jenkins.
- Strong knowledge of API design, gateway management, OAuth2, OIDC, service-to-service communication, IAM, secrets management, and cloud network security.
- Practical experience with metrics, logging, distributed tracing, and alerting tools such as Prometheus, Grafana, Datadog, AWS CloudWatch, or OpenTelemetry.
- Proficiency in Python or modern programming languages used for automation and platform tooling.
- Working knowledge of LLM APIs, foundation models, token-based usage models, model routing and gateways, streaming responses, rate limiting, and generative AI service operational constraints.
- Preferred experience includes Portkey or comparable enterprise AI gateways, LangSmith, Phoenix, Arize, Traceloop, AI-focused FinOps, and infrastructure services in large federated or higher-education environments.
Benefits
- Annual base salary range is USD $170,000.00 to $190,000.00.
- NYU is an equal opportunity employer and promotes equal treatment and opportunity in hiring.
- NYU aims to be among the greenest urban campuses in the country and carbon neutral by 2040.
Tech Stack
Categories
About New York University
New York University is a private research university that offers undergraduate, graduate, and professional degree programs for students worldwide and operates a major academic medical center. It generates revenue primarily from tuition, research grants, and clinical services, and runs global campuses in New York City, Abu Dhabi, and Shanghai with study-away sites like London. Founded in 1831 and headquartered in New York City, it supports extensive research computing, HPC, and secure data environments.