Senior Software Engineer, AI/ML (Infrastructure & Platform)

4 months ago

H1B sponsor

Base Salary

$200k - $250k/yr

Responsibilities

Design and implement platforms for LLM orchestration, tool execution, agent workflows, shared AI services, and reusable capability abstractions.
Build reliable, observable, and secure tools and skills for data retrieval, calculations, document processing, and external system actions.
Develop infrastructure for multi-step agents, including state management, tool routing, retries, failure handling, and model-to-capability orchestration.
Create evaluation, logging, tracing, and monitoring systems for AI behavior and production performance.
Design for high availability, fault tolerance, graceful degradation, latency, throughput, and cost efficiency across AI workloads.
Develop scalable RAG pipelines, indexing systems, data-processing workflows, and infrastructure for structured and unstructured data.
Build internal platforms, SDKs, and developer tooling that help engineers integrate AI capabilities safely and consistently.
Partner with AI Applications and product teams to translate product needs into scalable infrastructure solutions.

Degree in Computer Science, Engineering, or a related quantitative field, or equivalent practical experience.
Strong software engineering fundamentals in system design, distributed systems, and maintainable code.
Proven experience building and operating production systems at scale.
Proficiency in Python, TypeScript, and C#, with comfort working across a polyglot stack.
Experience building backend systems, APIs, or infrastructure platforms.
Production experience with AI/ML systems, including LLM integrations or data pipelines.
Experience designing or integrating tool, skill, function-calling, API, or capability abstractions for AI systems.
Preferred experience includes AI platforms, RAG systems, vector databases, agent orchestration frameworks, AI evaluation and observability tooling, scalable distributed systems, platform abstractions, cloud infrastructure, containerized or serverless deployments, event-driven systems, queues, asynchronous processing, MLOps, CI/CD, production monitoring, regulated domains, and data privacy or security techniques such as PII handling and redaction.

Hybrid work in the New York area.
Medical, dental, and vision insurance options with low-cost premium structures.
Company-paid basic life insurance and short- and long-term disability insurance.
Paid parental leave upon eligibility.
Company equity managed through Carta.
401(k) with match and 100% vesting upon hire.
Flexible PTO and paid holidays, including a birthday holiday.

201-500 employees