12 days ago
Chicago, IL, USA +4 moreSenior
Base Salary
$125k - $186k/yr
Responsibilities
- Drive AI innovation and rapid experimentation involving agentic workflows, reasoning approaches, evaluation methods, and emerging techniques.
- Design and execute timeboxed pilots with hypotheses, success metrics, and kill-or-scale decision points.
- Enhance enterprise AI patterns such as retrieval-augmented generation, text-to-SQL, evaluation and observability, and guardrails.
- Build reference implementations and handoff documentation for successful experiments in partnership with the Central AI team.
- Develop secure, observable, cost-aware AI architectures within the AWS ecosystem using strong software engineering practices.
- Design and interpret LLM evaluations covering task success, faithfulness, hallucination analysis, cost, and latency.
- Develop and evaluate agentic systems involving tool use, memory strategies, multi-agent orchestration, and reliability techniques.
- Communicate technical findings, business impact, tradeoffs, and recommendations through demos and concise readouts.
- Evaluate emerging AI tools, frameworks, vendor offerings, external research, and open-source trends.
- Support priority project work focused on de-risking and accelerating delivery.
Requirements
- 8+ years of software engineering experience or equivalent delivering production-quality systems.
- Expert Python skills including clean architecture, testing, packaging, performance, and reliability.
- Strong AWS architecture and development experience involving security/IAM, networking, serverless or containers, monitoring/logging, and cost controls.
- Strong SQL, data modeling, and API/integration-pattern experience, with the ability to incorporate RAG and text-to-SQL into real solutions.
- Demonstrated rigor in hypothesis-driven experimentation, evaluation planning, metrics, timeboxing, and pragmatic decision-making.
- Clear communication with technical and business stakeholders, including storytelling and influence through results.
- Bachelor’s degree in Computer Science or Engineering, or equivalent practical experience.
- Hands-on experience with modern LLM application stacks, abstraction tradeoffs, portability, and maintainability.
- Preferred: experience with Dataiku DSS and operationalizing analytics or AI workflows, with LLMOps experience as a plus.
- Preferred: experience building evaluation harnesses with golden sets, regression testing, and error analysis, and partnering on risk or safety reviews.
- Preferred: exposure to OpenAI AgentKit, ChatKit, Apps SDK, and MCP.
- Preferred: experience transitioning prototypes into enterprise-ready patterns with platform and delivery teams.
Benefits
- Full-time US roles include healthcare, dental, and vision insurance for employees and eligible dependents.
- 401(k) retirement plan with a company match of 50% up to 12% of eligible compensation.
- Generous PTO starting at 22 days per year, plus paid holidays and volunteer time.
- Additional wellness, financial, and work/lifestyle-specific benefits are offered.
- The role is full time and located in San Francisco, with additional locations in Chicago, Denver, Los Angeles, and Phoenix.
Categories
