2 hours ago
Carmel, IN, USAMid Level
Responsibilities
- Embed with business teams to identify valuable agent use cases and learn their processes firsthand.
- Translate business processes into agent requirements, scope, success criteria, and measurable outcomes.
- Design, build, pilot, and roll out safe, reliable AI agents with tool use, integrations, and multi-step workflow orchestration.
- Define and implement evaluations measuring AI quality, consistency, accuracy, and regression over time.
- Design human-oversight checkpoints for client-facing outputs and writes to systems of record.
- Develop reusable agent patterns, skills, templates, and best practices for company-wide development.
- Lead business-user piloting and acceptance testing before rollout.
- Partner with data scientists, business partners, and peer AI teams to prioritize use cases and coordinate delivery.
- Apply responsible-AI practices and stay current with agentic AI and evaluation methods.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related technical field, or equivalent practical experience.
- At least 3 years of total experience delivering AI, software, or data solutions, including hands-on LLM-based application or agent development.
- Experience designing AI agents with tool use, external system and data integration, and multi-step workflow orchestration.
- Strong generative AI and large language model proficiency, including prompt engineering.
- Experience implementing evaluations for AI systems that measure quality, consistency, and regression.
- Strong requirements-discovery, facilitation, communication, and stakeholder collaboration skills.
- Strong Python proficiency and hands-on cloud-platform experience; Azure is preferred, with AWS or GCP acceptable.
- Ability to work independently, manage multiple initiatives, operate in ambiguous greenfield environments, and keep current with rapidly changing technology.
- Experience with Model Context Protocol or similar frameworks and integrations with systems of record or CRM platforms such as Microsoft Dynamics is preferred.
- Customer-facing, consulting, solutions-engineering, or forward-deployed experience is preferred.
- Background in data science, machine learning, or applied statistics, especially evaluation or measurement methodology, is preferred.
Benefits
- Hybrid work model combining in-person collaboration and remote work, with scheduling details based on team and business needs.
- Health, dental, and vision insurance.
- Unlimited paid time off.
- 401(k) plan with a currently offered 6% company match and no vesting period.
- Health Savings Accounts and Flexible Spending Accounts.
- Company-paid short-term and long-term disability insurance.
- Term life insurance with optional supplemental coverage.
- Tuition reimbursement.
- Voluntary wellness program with up to $2,000 in rewards.
- Employee discounts through Perks at Work.
- Community involvement and giving-back opportunities.
- CliftonStrengths assessment and coaching opportunities.
Tech Stack
Categories
