Senior/Principal Local LLM & Generative AI Platform Engineer
Parallel Wireless4 months ago
Kefar Sava, IsraelSenior / Staff+
Responsibilities
- Own the architecture and technical roadmap for a secure, reliable local LLM platform deployed in company-controlled infrastructure.
- Partner with engineering, product, support, IT, information security, legal, and domain experts to prioritize use cases and define measurable requirements.
- Build model gateways and inference services with stable APIs, model routing, streaming, concurrency controls, quotas, and interchangeable serving backends.
- Evaluate and optimize open-weight language, code, embedding, reranking, and multimodal models across CPU, GPU, and accelerator resources.
- Design and operate RAG and enterprise-search pipelines with parsing, chunking, metadata, embeddings, hybrid retrieval, reranking, citations, freshness, and deletion controls.
- Enforce source-system permissions using identity, SSO, role-based access control, secrets management, and audit logging.
- Create versioned evaluation datasets, automated quality and safety evaluations, release gates, regression tests, canary releases, and rollback processes.
- Implement observability for model and agent workflows, including latency, throughput, resource utilization, availability, errors, traces, and user feedback.
- Design safe tool-calling and agent workflows with least privilege, sandboxing, validation, bounded execution, human approval, and traceability.
- Integrate the platform with developer environments, source control, CI workflows, knowledge systems, ticketing systems, and internal applications through SDKs and APIs.
- Build production foundations including CI/CD, registries, backups, disaster recovery, capacity planning, vulnerability management, incident response, and lifecycle policies.
- Protect proprietary and personal information through isolation, encryption, retention controls, redaction, secure logging, and defenses against prompt injection, data poisoning, unsafe outputs, and model supply-chain risks.
- Determine when prompting or retrieval improvements are sufficient and when fine-tuning, distillation, or other model adaptation is justified.
- Improve platform adoption through documentation, examples, training, office hours, telemetry, feedback, and cross-functional collaboration.
Requirements
- BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; equivalent depth may be accepted.
- Strong Python engineering skills and experience designing maintainable APIs, services, libraries, and data pipelines.
- Strong understanding of transformer-based language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, tool calling, and model failure modes.
- Experience building production RAG or enterprise-search systems with embeddings, vector or lexical search, metadata filtering, reranking, source attribution, and systematic retrieval evaluation.
- Experience defining task-specific LLM evaluations using representative datasets, baselines, expert review, automated metrics, human feedback, error analysis, and regression thresholds.
- Experience deploying and operating containerized services on Linux using Docker and Kubernetes or an equivalent orchestration environment.
- Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.
- Strong knowledge of distributed systems, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls.
- Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response.
- Ability to lead an ambiguous cross-functional initiative, explain complex AI behavior clearly, and help teams ship safely on a shared platform.
- Experience with Go, Java, or C/C++ is advantageous.
- Experience with on-premises, private-cloud, restricted-network, or air-gapped LLM deployments is preferred.
- Experience with inference runtimes such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, or Triton is preferred.
- Experience optimizing NVIDIA or AMD GPU inference with CUDA, ROCm, profiling, parallelism, speculative decoding, or prefix/KV caching is preferred.
- Experience with model and experiment registries, LLM tracing and evaluation platforms, vector databases, hybrid-search engines, and production data-orchestration frameworks is preferred.
- Experience with LoRA or QLoRA, dataset curation, synthetic-data generation, distillation, and post-training evaluation is preferred.
- Experience building code intelligence, repository-aware assistants, developer tools, or IDE and CI integrations for large C/C++ and Python codebases is preferred.
- Familiarity with Active Directory, fine-grained document authorization, data-loss prevention, secure software supply chains, model licensing, and AI governance is preferred.
- Experience red-teaming LLM or agent systems is preferred; telecommunications, 3GPP, RAN/Open RAN, cloud-native network functions, technical-support workflows, heterogeneous compute, and relevant open-source contributions are additional advantages.
Categories
About Parallel Wireless
Parallel Wireless builds open, cloud-native RAN software and solutions for 2G–5G networks used by mobile network operators. Its ALL G O-RAN platform supports interoperable deployments and network automation, with software licensing and services as key revenue streams. Founded in 2012 and headquartered in Nashua, New Hampshire, the privately held company reports engagements with over 50 mobile network operators worldwide.