Senior AI Engineer — Inference & Agent Systems
Arcana Analytics8 months ago
USA (city not specified)Senior / Staff+
H1B sponsor
Responsibilities
- Drive time-to-first-token (TTFT) below 400ms for multi-step agent pipelines.
- Implement streaming optimization to deliver the first token to users while sub-agents are still running.
- Develop KV cache strategies, prompt compression, and dynamic context window management.
- Design and implement Plan-Execute-Synthesize pipelines for parallel execution of sub-agents.
- Build reliable orchestration using Temporal, including retries and partial failure recovery.
- Own the evaluation framework, including ground truth datasets and automated scoring pipelines.
- Conduct latency regression testing and design adversarial test cases for robustness.
- Optimize model serving and cold start processes, ensuring observability of every token and tool call.
Requirements
- Experience building production systems with a focus on performance and latency.
- Proven track record in optimizing inference pipelines with measurable improvements.
- Hands-on experience with multi-step agent systems and understanding of their failure points.
- Ability to create evaluation harnesses and insights into effective ground truth datasets.
- Familiarity with streaming LLM responses and infrastructure for handling partial outputs.
- Proficiency in Go, Python, Temporal, Kafka, PostgreSQL, and Docker.
Tech Stack
Categories
Data Engineering
About Arcana Analytics
Arcana Analytics builds a portfolio intelligence platform for hedge funds and asset managers, combining portfolio data with analytics on performance attribution, factor risk, crowding, and ownership using proprietary datasets. It sells enterprise software and data subscriptions used for portfolio construction, real-time risk, and research workflows by institutional investors in global markets, including APAC. Founded in 2022 and headquartered in New York, the company is privately held and raised a Series B in 2026.