Senior/Principal Local LLM & Generative AI Platform Engineer
Parallel Wireless2 months ago
USA (city not specified)Senior / Staff+
H1B sponsor
Responsibilities
- Own the architecture, roadmap, and operation of a secure and maintainable local LLM platform.
- Build model gateways with stable APIs, routing, streaming, concurrency controls, quotas, and interchangeable serving backends.
- Evaluate, select, and optimize open-weight language, code, embedding, reranking, and multimodal models for company-specific workloads.
- Design and operate RAG and enterprise-search pipelines with parsing, chunking, embeddings, hybrid retrieval, reranking, citations, freshness, deletion, and permission enforcement.
- Establish automated offline and online evaluations for retrieval, groundedness, factual accuracy, citations, code correctness, latency, safety, refusals, and task completion.
- Implement regression testing, release gates, canary releases, rollback procedures, and approval paths for model, prompt, tool, retrieval, index, and serving changes.
- Build observability for LLM and agent workflows, including traces, latency, throughput, resource utilization, saturation, availability, and feedback.
- Design safe tool-calling and agent workflows with least privilege, sandboxing, validation, bounded execution, human approval, and traceability.
- Integrate the platform with developer environments, source control, CI workflows, knowledge systems, ticketing systems, and internal applications.
- Establish production operations including CI/CD, registries, backups, disaster recovery, capacity planning, vulnerability management, incident response, and lifecycle policies.
- Protect proprietary and personal information through isolation, encryption, retention controls, redaction, secure logging, and defenses against prompt injection, data poisoning, unsafe outputs, and model-supply-chain risks.
- Make build-versus-buy, prompting, retrieval, fine-tuning, distillation, hardware-efficiency, quality, security, and total-cost tradeoffs.
- Enable adoption through documentation, examples, training, office hours, cross-functional collaboration, and structured feedback.
Requirements
- Bachelor’s or master’s degree in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems.
- Strong Python skills and experience designing maintainable APIs, services, libraries, and data pipelines; Go, Java, or C/C++ experience is advantageous.
- Strong understanding of transformer language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, tool calling, and model failure modes.
- Experience building production RAG or enterprise-search systems using embeddings, vector or lexical search, metadata filtering, reranking, source attribution, and retrieval evaluation.
- Experience defining task-specific LLM evaluations using representative datasets, expert review, automated metrics, human feedback, error analysis, and regression thresholds.
- Experience deploying and operating containerized Linux services using Docker and Kubernetes or equivalent orchestration.
- Experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.
- Strong knowledge of distributed systems, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls.
- Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response.
- Ability to lead an ambiguous cross-functional initiative, communicate complex AI behavior clearly, and help teams ship safely on a shared platform.
- Preferred experience includes private or air-gapped LLM deployments; inference systems such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, or Triton; NVIDIA or AMD GPU optimization; model registries, evaluation platforms, vector databases, search engines, and data orchestration; LoRA/QLoRA, distillation, code intelligence, enterprise identity, AI governance, LLM red-teaming, telecommunications, and open-source contributions.
Categories
About Parallel Wireless
Parallel Wireless builds open, cloud-native RAN software and solutions for 2G–5G networks used by mobile network operators. Its ALL G O-RAN platform supports interoperable deployments and network automation, with software licensing and services as key revenue streams. Founded in 2012 and headquartered in Nashua, New Hampshire, the privately held company reports engagements with over 50 mobile network operators worldwide.