Senior LLMOps Engineer, Development (52647)
Citrin Cooperman22 hours ago
Remote, United StatesSenior
Base Salary
$155k - $195k/yr
Responsibilities
- Design and build CI/CD pipelines for deploying generative AI applications, prompts, LangGraph state machines, and RAG retrieval logic across Dev, Test, and Prod environments.
- Deploy and manage continuous AI evaluation infrastructure with precision, recall, toxicity, and other automated deployment-gate checks.
- Instrument AI applications and build dashboards for token consumption, latency, reasoning traces, API failures, and other operational metrics across multiple LLM providers.
- Implement version control and auditable registries for prompts and model configurations.
- Integrate input and output guardrails to address prompt injection, PII leakage, and off-topic responses.
- Monitor AI solution costs, alert on token-usage spikes, and optimize embedding and retrieval strategies for efficiency.
Requirements
- Bachelor’s degree in computer science, information technology, engineering, or equivalent practical experience.
- Databricks Certified: Machine Learning Professional certification.
- Microsoft Certified: Azure DevOps Engineer Expert (AZ-400) certification.
- DeepLearning.AI: Machine Learning Engineering for Production (MLOps) certification.
- At least 4 years of DevOps, MLOps, or Site Reliability Engineering experience, including hands-on generative AI deployment experience within the last 1–2 years.
- Proficiency building CI/CD pipelines with Azure DevOps, GitHub Actions, or GitLab CI.
- Hands-on experience with LLMOps tools and frameworks such as MLflow, LangSmith, PromptFlow, or Arize.
- Strong Python scripting skills and experience containerizing machine learning or API workloads with Docker and Kubernetes.
- Understanding of frontier-model API ecosystems including OpenAI, Anthropic, and Google Vertex AI, plus multi-agent frameworks such as LangChain and LangGraph.
- Familiarity with Azure, AWS, and infrastructure-as-code principles.
- Strong automation, financial monitoring, troubleshooting, and analytical problem-solving orientation.
Benefits
- Competitive compensation and benefits are offered.
- Flexible work environment intended to support personal and professional life and professional development.
Tech Stack
Categories
About Citrin Cooperman
Citrin Cooperman is a privately held professional services firm providing audit, tax, accounting, and advisory services—including transaction advisory, valuation/forensics, wealth management, and cybersecurity—to middle-market companies and high-net-worth individuals. Founded in 1979 and headquartered in New York, it operates under an alternative practice structure (Citrin Cooperman & Company, LLP and Citrin Cooperman Advisors LLC) and is backed by Blackstone. The firm has 30+ offices and is a member of the Moore Global network.