18 hours ago
Toronto, CanadaSenior
Responsibilities
- Design and implement an enterprise-grade agent platform with orchestration patterns, tool execution, context and memory management, safety guardrails, governance, permission isolation, and secure execution controls.
- Build reusable agent skills, standardized tool interfaces, and scalable integrations with NetBrain platform capabilities and business workflows.
- Develop LLM post-training strategies including supervised fine-tuning, DPO/RLHF preference alignment, and parameter-efficient fine-tuning.
- Create self-learning feedback loops using production traces, user feedback, and evaluation results to improve prompts, skills, models, and retrieval.
- Build LLM and agent evaluation frameworks, benchmark datasets, regression pipelines, quality gates, hallucination detection, and task-success metrics.
- Implement AI observability with distributed tracing, structured logging, metrics, dashboards, and alerting, and resolve production failures and regressions.
- Design reliable backend services for AI workloads, including asynchronous and concurrent processing, retries, timeouts, caching, rate limiting, and fault isolation.
- Optimize latency, throughput, token consumption, and infrastructure costs while meeting platform SLA requirements.
- Prototype, benchmark, and productionize emerging agent, retrieval, and LLM technologies and contribute to platform architecture and technology strategy.
- Lead technical design for critical modules and collaborate with Engineering, Product, QA, and other teams to deliver production solutions.
Requirements
- Bachelor's degree or higher in Computer Science, Artificial Intelligence, Electrical Engineering, or a related technical field, or equivalent practical experience.
- At least 3 years of experience in software engineering, machine learning, or applied AI, including at least 2 years building, deploying, and operating production-grade LLM or agent applications.
- Experience delivering at least one LLM-powered feature end-to-end and owning its operation and improvement after production launch.
- Deep understanding of agent architectures and LLM behavior, with hands-on experience in multi-step workflows involving reasoning, tools, state management, structured outputs, validation, and error recovery.
- Experience diagnosing and resolving production LLM and agent failures such as hallucinations, incorrect tool calls, retrieval degradation, agent loops, structured-output failures, and latency regressions.
- Strong Python and distributed backend engineering skills, including API and service development, asynchronous and concurrent programming, testing, logging, and cross-service performance debugging.
- Experience designing LLM evaluation systems, including dataset construction, metric definition, regression testing, and release quality gates.
- Understanding of LLM and agent security risks including prompt injection, data leakage, unsafe tool execution, permission boundaries, and uncontrolled autonomy.
- Ability to independently design, implement, debug, deploy, and operate complex production systems.
- Preferred experience with RAG, embeddings, vector and hybrid search, reranking, chunking, grounding, citation mechanisms, multi-hop retrieval, Knowledge Graphs, or GraphRAG.
- Preferred familiarity with LangGraph, LangChain, AutoGen, LlamaIndex, MCP, agent runtime mechanisms, human-in-the-loop workflows, long-running workflow context management, and memory systems.
- Preferred experience with LangSmith or similar LLM observability and evaluation platforms and with LoRA or other parameter-efficient fine-tuning techniques.
- Experience applying LLM technologies to networking, infrastructure, cybersecurity, observability, or other complex technical domains is preferred.
- Fluency in English and Chinese with strong cross-regional communication and collaboration skills is preferred.
Benefits
- Estimated base salary is CAD $130,000–CAD $165,000 plus bonus.
- Benefits include RRSP and medical/dental coverage.
- Additional benefits and total rewards details are available from the recruiter.
Categories
About NetBrain
NetBrain builds network automation and operations software for large enterprises and service providers, combining no-code runbooks and AI-driven diagnostics to map, troubleshoot, and safeguard changes across hybrid multi-cloud networks. The company sells subscriptions and licenses for its Next-Gen platform and tools for network discovery, documentation, and change management; it is privately held, founded in 2004, and headquartered in Burlington, Massachusetts. Its software is used by thousands of global enterprises and MSPs.