Staff Engineer / AI Builder - Toronto
Robots and Pencils11 hours ago
Toronto, CanadaStaff+
Base Salary
$166k - $228k/yr
Responsibilities
- Design and build single- and multi-agent workflows with reasoning loops, tool/function calling, and orchestration.
- Build and maintain RAG pipelines covering chunking, embeddings, vector search, re-ranking, and knowledge-source refresh handling.
- Integrate AWS Bedrock and Agent Core, including MCP-based tool design and production system prompts.
- Build LLM evaluation and observability systems using golden datasets, RAGAS-style metrics, LLM-as-judge techniques, and tracing.
- Design reliability mechanisms including retries, backoff, jitter, circuit breakers, fallback models, and user-facing degradation handling.
- Build Python and Node.js backend services, including AWS Lambda and API Gateway serverless architectures.
- Design DynamoDB single-table schemas for conversation state, agent memory, and session history.
- Implement event-driven orchestration with Step Functions, SQS, and EventBridge.
- Contribute to frontend integration, deployment, monitoring, troubleshooting, and cloud-native production operations.
- Lead architecture discussions, partner with product, design, and delivery teams, mentor engineers, and own features and releases end-to-end.
Requirements
- 6+ years of professional software engineering experience, including meaningful production experience shipping GenAI/LLM-powered systems.
- Hands-on experience with agentic AI, reasoning loops, tool/function calling, and single- and multi-agent orchestration.
- Practical RAG expertise involving chunking strategies, embeddings, vector databases, cosine similarity search, and re-ranking.
- Experience with LLM evaluation and observability, including golden datasets, LLM-as-judge, RAGAS or comparable metrics, and tracing tools.
- Strong production prompt-engineering skills, including structured system prompts, output constraints, and few-shot design.
- Hands-on experience with AWS Bedrock, Agent Core, Lambda, DynamoDB single-table design, S3, SQS, EventBridge, and Step Functions.
- Strong Python and Node.js skills with experience building full-stack applications and APIs.
- Understanding of production reliability patterns for LLM-backed systems, including retries, backoff, circuit breakers, and fallback models.
- Experience with Docker and cloud-native deployment.
- Ability to own ambiguous integration-heavy problems and make informed architectural trade-offs.
- Legally authorized to work in Canada and eligible to secure immediate U.S. non-immigrant work authorization, such as TN status under USMCA, for mandatory U.S. client delivery.
- Experience with Amazon Bedrock Agent Core, including agent registration, MCP tool wiring, memory/session management, and gateway provisioning is helpful.
- Experience in regulated or high-stakes domains, familiarity with Step Functions, Sequencer, or similar workflow tools, and production LLM observability experience are helpful.
Benefits
- Comprehensive benefits package including paid time off, medical, dental, and vision insurance, plus a 401(k) for eligible employees.
- Mandatory weekly on-site client delivery in the United States three days per week.
- Employment may be conditional upon successful completion of a background check in accordance with local legislation.
About Robots and Pencils
Robots & Pencils is an applied AI engineering firm that designs, builds, and runs production AI systems and agentic co-workers for enterprise operations, with a focus on AWS. The privately held company, founded in 2009 and headquartered in Chagrin Falls, Ohio, provides consulting and implementation services spanning data readiness, cloud modernization, and AI enablement. It is an AWS Advanced Tier Services Partner with delivery teams across the U.S., Canada, Eastern Europe, and Latin America.