AI Agent Engineer
Scout Motors4 hours ago
Charlotte, NC, USAMid Level
H1B sponsor
Base Salary
$95k - $140k/yr
Responsibilities
- Design, develop, deploy, version, monitor, and maintain multiple AI agents on Databricks.
- Build and customize MLflow evaluation frameworks and judges for correctness, safety, and business-specific criteria.
- Run offline dataset-based evaluations and online production monitoring, using MLflow tracing to analyze agent execution.
- Collect stakeholder feedback and incorporate it into agent evaluation and improvement cycles.
- Collaborate with Data Engineering, IT, and R&D on data configurations and pipelines for AI agents.
- Combine on-site, off-site, and connected vehicle data to identify trends, anomalies, and quality signals.
- Create dashboards and visual reports that communicate agent outputs and quality insights to business stakeholders.
- Define requirements and automated checks to identify and close data-quality gaps.
Requirements
- Bachelor's degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, or a related field, or equivalent practical experience.
- At least 4 years of experience in MLOps, AI/ML engineering, or production data engineering.
- Experience designing evaluation workflows for non-deterministic AI systems using offline datasets and online production monitoring.
- Understanding of LLM quality concepts including hallucination, grounding, retrieval quality, tool-use reliability, and safety.
- Hands-on experience developing and deploying AI agents on Databricks, including Mosaic AI, the Databricks Agent Framework, multi-agent architectures, and lifecycle management.
- Deep expertise with MLflow Tracking, Model Registry, Deployments, evaluation frameworks, judges, and tracing.
- Strong Python skills and experience with agent development frameworks such as LangChain, LlamaIndex, or AutoGen.
- Proficiency in SQL and cloud-based storage systems such as Delta Lake or Unity Catalog.
- Familiarity with CI/CD practices for ML systems, including model versioning, automated testing pipelines, and deployment gates.
- Experience with BI tools such as Sigma or Power BI is preferred.
- Experience with RAG pipelines, vector stores, embedding models, and chunking strategies is preferred.
- Familiarity with Streamlit or Gradio and responsible AI or AI governance frameworks is preferred.
- Strong communication, structured problem-solving, independent execution, collaboration, and adaptability in fast-moving environments.
Benefits
- Medical, dental, vision, and income protection insurance plans.
- 401(k) program with employer match and immediate vesting.
- 20 days of planned PTO as accrued, 40 hours of unplanned PTO, and 14 company or floating holidays annually.
- Up to 16 weeks of paid parental leave for biological and adoptive parents of all genders.
- Paid leave for bereavement, jury duty, voting time, and military leave.
- Full-time exempt position based at the Scout Motors headquarters in Charlotte, NC, requiring 4–5 days per week in the office and occasional travel.
Categories
About Scout Motors
Scout Motors designs and manufactures electric trucks and rugged SUVs for consumer drivers, reviving the historic Scout off‑road brand. Founded in 2022 and headquartered in Charlotte, North Carolina, the company is building a manufacturing plant in Blythewood, South Carolina. Its lineup targets work and outdoor use, with both an all‑electric powertrain and a Harvester range‑extender option that integrates a gas generator for extended range.