Staff AI Builder
Robots and Pencils5 hours ago
Responsibilities
- Design and build agentic workflows using reasoning loops, tool/function calling, and single- and multi-agent architectures.
- Build and maintain RAG pipelines covering chunking, embeddings, vector search, re-ranking, and knowledge-source refresh handling.
- Integrate AWS Bedrock and Agent Core, including MCP-based tool design.
- Write production system prompts with structured role framing, output constraints, and few-shot design.
- Build evaluation and observability systems using golden datasets, RAGAS-style metrics, LLM-as-judge, and tracing tools.
- Design reliability and failure-handling patterns including retries, backoff, jitter, circuit breakers, fallback models, and user-facing degradation behavior.
- Build Python and Node.js backend services and AWS Lambda/API Gateway serverless architectures.
- Design DynamoDB single-table schemas for conversation state, agent memory, and session history.
- Support event-driven orchestration with Step Functions, SQS, and EventBridge.
- Contribute to frontend integration, deployment, monitoring, production troubleshooting, and clean tested code across the stack.
- Lead architecture discussions, partner with product, design, and delivery teams, mentor engineers, and own features and releases end-to-end.
Requirements
- 6+ years of professional software engineering experience, including meaningful production experience shipping GenAI/LLM-powered systems.
- Hands-on expertise in agentic AI, including reasoning loops, tool/function calling, and multi-agent orchestration.
- Practical RAG experience with chunking strategies, embeddings, vector databases, cosine similarity search, and re-ranking.
- Experience building LLM evaluation and observability systems using golden datasets, LLM-as-judge, RAGAS or comparable metrics, and tracing tools.
- Strong production prompt-engineering skills involving structured prompts and constraints.
- Hands-on experience with AWS Bedrock, Agent Core, Lambda, DynamoDB single-table design, S3, SQS, EventBridge, and Step Functions.
- Strong Python and Node.js skills, including full-stack application and RESTful API development.
- Understanding of production reliability patterns for LLM-backed systems, including retries/backoff, circuit breakers, and fallback models.
- Experience with Docker, cloud-native deployment, ambiguous integration-heavy problems, and end-to-end ownership.
- Helpful qualifications include Amazon Bedrock Agent Core experience, regulated-domain experience, familiarity with Sequencer or similar state-machine orchestration tools, and production experience with LLM observability tooling.
Benefits
- The role supports global cross-functional teams and production delivery for enterprise clients.
- The company ships production-ready AI systems in 30 to 45 days.
- Salary range is CAD $176,612–$243,680.
Categories
About Robots and Pencils
Robots & Pencils is an applied AI engineering firm that designs, builds, and runs production AI systems and agentic co-workers for enterprise operations, with a focus on AWS. The privately held company, founded in 2009 and headquartered in Chagrin Falls, Ohio, provides consulting and implementation services spanning data readiness, cloud modernization, and AI enablement. It is an AWS Advanced Tier Services Partner with delivery teams across the U.S., Canada, Eastern Europe, and Latin America.