AI Agent Engineer
Scout Motors17 days ago
Charlotte, NC, USAMid Level
H1B sponsor
Base Salary
$95k - $140k/yr
Responsibilities
- Design, develop, deploy, version, monitor, and maintain multiple AI agents on Databricks across their full lifecycle.
- Build and customize offline and online AI-agent evaluation frameworks using MLflow evaluation tools, built-in judges, guideline-based judges, and custom judges.
- Improve agent quality using curated datasets, benchmarks, production monitoring, MLflow tracing, and structured human feedback.
- Collaborate with Data Engineers, IT, R&D, engineering, quality, and business stakeholders on data configurations, evaluation workflows, and agent improvement.
- Combine on-site, off-site, and connected vehicle data to identify trends, anomalies, and quality signals at scale.
- Present agent outputs and quality insights through dashboards and visual reporting tools for Quality stakeholders.
- Identify and close data quality gaps through defined requirements and automated checks.
Requirements
- Bachelor's degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, or a related field, or equivalent practical experience.
- At least 4 years of experience in MLOps, AI/ML engineering, or production data engineering.
- Experience designing and executing offline and online evaluation workflows for non-deterministic AI systems.
- Understanding of LLM quality concepts including hallucination, grounding, retrieval quality, tool-use reliability, and safety.
- Hands-on experience developing and deploying AI agents on Databricks, including Mosaic AI, the Databricks Agent Framework, multi-agent architectures, and agent lifecycle management.
- Deep expertise with MLflow Tracking, Model Registry, Deployments, evaluation frameworks, judges, and tracing for agent observability.
- Strong Python proficiency and experience with agent development frameworks such as LangChain, LlamaIndex, or AutoGen.
- Proficiency in SQL and cloud-based storage systems such as Delta Lake or Unity Catalog.
- Familiarity with CI/CD practices for ML systems, model versioning, automated testing pipelines, and deployment gates.
- Experience with BI tools such as Sigma or Power BI is preferred.
- Experience building RAG pipelines with vector stores, embedding models, and chunking strategies is preferred.
- Familiarity with Streamlit or Gradio and knowledge of responsible AI or AI governance frameworks are preferred.
- Strong communication, structured problem-solving, independent execution, and adaptability in multidisciplinary environments.
Benefits
- Medical, dental, vision, and income protection insurance plans.
- 401(k) with employer match and immediate vesting.
- 20 days of planned PTO as accrued, 40 hours of unplanned PTO, and 14 company or floating holidays annually.
- Up to 16 weeks of paid parental leave for biological and adoptive parents of all genders.
- Paid leave for bereavement, jury duty, voting time, and military leave.
- Full-time, exempt position with an annual performance bonus program.
- Based at Scout Motors' Charlotte, North Carolina headquarters, with four to five days per week in the office and occasional travel.
- The role is not eligible for remote work in New York City.
Categories
About Scout Motors
Scout Motors designs and manufactures electric trucks and rugged SUVs for consumer drivers, reviving the historic Scout off‑road brand. Founded in 2022 and headquartered in Charlotte, North Carolina, the company is building a manufacturing plant in Blythewood, South Carolina. Its lineup targets work and outdoor use, with both an all‑electric powertrain and a Harvester range‑extender option that integrates a gas generator for extended range.