AI Engineer- NLP
Velocitor Solutions1 month ago
Charlotte, NC, USAMid Level / Senior
Responsibilities
- Extend the guardrails, query reformulation, embedding-based routing, LangGraph agent, response formatting, and follow-up generation pipeline.
- Maintain and expand the VTrack API domain tool layer, including schemas, authorization, pagination, date handling, and error formatting.
- Support the production service by responding to incidents, investigating latency and quality regressions, and improving telemetry and runbooks.
- Improve RAG retrieval quality using PostgreSQL full-text search, pgvector, hybrid retrieval, and reranking where appropriate.
- Build evaluation gates for model and prompt changes using representative and adversarial datasets, shadow traffic, canary rollout, and automated rollback.
- Reduce LLM cost and latency through token and cost telemetry, prompt and context trimming, caching, model tiering, and removal of redundant LLM stages.
- Strengthen tenant-scoped authorization, prompt-injection defenses, and security boundaries across prompts, retrieval, tools, authorization, and outputs.
- Extend tiered testing across commit, pull request, nightly, and release gates.
Requirements
- Three or more years of experience building and supporting production backend services, including hands-on delivery of at least one LLM-backed feature used by real users.
- Strong Python skills, including asynchronous programming, type-driven design, and work in a strict mypy codebase.
- Experience with an LLM orchestration or agent framework such as LangChain, LangGraph, or an equivalent, including tool and function calling.
- Solid PostgreSQL experience covering schema design, query performance, migrations, and connection-pool behavior under load.
- Production service support experience, including telemetry-based diagnosis, blast-radius assessment, and rollback decisions.
- Understanding of when to use autonomous agents versus deterministic workflows, especially for data-modifying or compliance-sensitive operations.
- Understanding of AI-system security boundaries, untrusted model and retrieved content, external authorization enforcement, and tenant-scoped data access.
- Ability to debug across service boundaries using traces, latency metrics, and correlation IDs.
- Familiarity with retries, jittered backoff, circuit breakers, and concurrency limits for rate-limited upstream providers.
- Testing experience beyond unit tests, including external API contract tests and evaluation of nondeterministic components.
- Prior production-system handover experience is preferred.
- Azure experience, especially Azure Container Apps, Azure OpenAI deployments and quota management, and Key Vault, is preferred.
- Terraform or OpenTofu, Azure DevOps Pipelines, vector search and RAG at scale, LLM-as-judge evaluation, and NeMo Guardrails or equivalent safety-layer experience are preferred.
- Modern React and TypeScript experience and data-retention or privacy-engineering experience are preferred.
Tech Stack
Categories