2 years ago
Arlington, VA, USASenior
Responsibilities
- Deploy and manage machine learning models in production using MLflow, Kubeflow, AWS SageMaker, or comparable platforms.
- Build monitoring and observability dashboards to track model health, accuracy, latency, and historical trends.
- Implement data drift detection pipelines and alerts using tools such as Evidently AI or Alibi Detect.
- Set up centralized logging and tracing for inference events, errors, and audit trails.
- Develop CI/CD pipelines to automate model updates, testing, and deployment.
- Apply secure-by-design practices, encryption, access controls, and applicable compliance requirements.
- Collaborate with data scientists, AI Integration Engineers, and DevOps teams on model performance and infrastructure needs.
- Optimize production models and cloud resource usage through techniques such as quantization and pruning.
- Document pipelines, dashboards, and monitoring processes.
Requirements
- Bachelor’s or master’s degree in Computer Science, Data Science, Engineering, or a related field.
- At least 5 years of experience in MLOps, DevOps, or software engineering focused on AI/ML systems.
- Experience deploying production models with MLflow, Kubeflow, AWS SageMaker, Azure ML, or comparable cloud platforms.
- Hands-on experience with Prometheus, Grafana, Datadog, or similar observability tools.
- Proficiency in Python and SQL; familiarity with JavaScript or Go is preferred.
- Expertise with Docker, Kubernetes, GitHub Actions, and Jenkins.
- Knowledge of InfluxDB, TimescaleDB, ELK Stack, OpenTelemetry, Evidently AI, Alibi Detect, Plotly, and Seaborn.
- Understanding of model performance metrics, drift detection methods, AI vulnerabilities, and mitigation tools such as Adversarial Robustness Toolbox.
- Preferred experience with LLM monitoring tools such as LangSmith or Helicone.
- Knowledge of GDPR and HIPAA compliance frameworks is preferred.
- Must be eligible to obtain a Department of Homeland Security EOD clearance, requiring U.S. citizenship and a favorable background investigation.
Tech Stack
AWSAzureDatadogDockerGitHub ActionsGoGoogle CloudGrafanaInfluxDBJavaScriptJenkinsKibanaKubernetesMLflowPrometheusPythonSeabornSQL
Categories
