Staff AI Builder
Robots and Pencils5 hours ago
Remote, CanadaStaff+
Responsibilities
- Design and build single- and multi-agent workflows with reasoning loops, tool/function calling, and orchestration.
- Build and maintain RAG pipelines covering chunking, embeddings, vector search, re-ranking, and knowledge-source refresh handling.
- Integrate AWS Bedrock and Agent Core, including MCP-based tool design.
- Write production system prompts with structured role framing, output constraints, and few-shot examples.
- Build evaluation and observability systems using golden datasets, RAGAS-style metrics, LLM-as-judge, and tracing.
- Design reliability mechanisms including retries, backoff, jitter, circuit breakers, fallback models, and user-facing degradation handling.
- Build backend and serverless services in Python and Node.js using AWS Lambda and API Gateway.
- Design DynamoDB single-table schemas for conversation state, agent memory, and session history.
- Support event-driven orchestration with Step Functions, SQS, and EventBridge.
- Contribute to frontend integrations and write clean, tested full-stack code.
- Support deployment, monitoring, and production troubleshooting in AWS and Docker environments.
- Lead architecture discussions, mentor engineers, collaborate with cross-functional teams, and own features and releases end-to-end.
Requirements
- 6+ years of professional software engineering experience.
- Meaningful professional experience shipping GenAI or LLM-powered systems in production.
- Hands-on expertise in agentic AI, reasoning loops, tool/function calling, and multi-agent orchestration.
- Practical RAG expertise including chunking strategies, embeddings, vector databases, cosine similarity search, and re-ranking.
- Experience with LLM evaluation and observability, including golden datasets, LLM-as-judge, RAGAS or comparable metrics, and tracing tools.
- Strong production prompt engineering skills.
- Hands-on experience with the AWS GenAI stack, including Bedrock, Agent Core, Lambda, DynamoDB, S3, SQS, EventBridge, and Step Functions.
- Strong Python and Node.js skills, including full-stack application and RESTful API development.
- Understanding of production reliability patterns for LLM-backed systems, including retries/backoff, circuit breakers, and fallback models.
- Experience with Docker, containerization, and cloud-native deployment.
- Ability to own ambiguous, integration-heavy problems and explain technical decisions in depth.
- Helpful qualifications include Amazon Bedrock Agent Core experience, regulated or high-stakes domain experience, workflow orchestration experience with Step Functions or Sequencer, and production experience with LLM observability tooling.
Categories
About Robots and Pencils
Robots & Pencils is an applied AI engineering firm that designs, builds, and runs production AI systems and agentic co-workers for enterprise operations, with a focus on AWS. The privately held company, founded in 2009 and headquartered in Chagrin Falls, Ohio, provides consulting and implementation services spanning data readiness, cloud modernization, and AI enablement. It is an AWS Advanced Tier Services Partner with delivery teams across the U.S., Canada, Eastern Europe, and Latin America.