Senior Agentic AI Engineer - Core Platforms
American Express5 days ago
Base Salary
$123k - $215k/yr
Responsibilities
- Design and maintain evaluation frameworks, benchmarks, scorecards, and success criteria for LLM, RAG, agentic, and multimodal AI solutions.
- Build automated and human-in-the-loop evaluation pipelines using curated datasets, synthetic test generation, human review, model-based grading, and deterministic checks.
- Develop regression testing, release gates, and continuous evaluation to detect quality degradation, drift, unsafe behavior, and system changes.
- Integrate AI models into production systems and build scalable inference services, retrieval pipelines, orchestration layers, and evaluation frameworks.
- Develop reusable Big Data pipelines, MLOps workflows, and end-to-end AI systems for deployment, monitoring, and maintenance.
- Translate business, product, risk, governance, and policy requirements into measurable acceptance criteria and traceable evidence.
- Perform failure analysis and communicate remediation priorities, tradeoffs, and actionable findings to technical and business stakeholders.
- Maintain adherence to model governance, security, compliance, reliability, safety, explainability, and observability standards.
- Mentor junior AI engineers and collaborate with product, engineering, risk, governance, and business teams.
Requirements
- At least 5 years of software development experience with hands-on experience in machine learning, AI technologies, and agentic solutions.
- Bachelor’s degree in computer science, machine learning, data science, or comparable experience; an advanced degree is preferred.
- Knowledge of Python and data processing technologies such as Hadoop and Spark.
- Knowledge of NLP techniques, fuzzy wuzzy, BERT, Transformer models, and real-world model applications.
- Advanced knowledge of machine learning techniques, performance optimization, AI system architectures, LLM ecosystems, retrieval systems, orchestration frameworks, and evaluation methodologies.
- Experience with generative AI, Big Data, prompt engineering, LLMs such as GPT-3.5/4, and machine learning algorithms.
- Experience with Python, R, Java, or C++.
- Experience deploying AI models and pipelines, building data pipelines and ETL processes, and delivering end-to-end AI systems in production.
- Experience with continuous integration and deployment pipelines, containerized deployments, distributed systems, scalable APIs, cloud-native AI infrastructure, monitoring, and optimization.
- Experience building evaluation harnesses and automated test pipelines for LLM, RAG, or agentic applications using production-representative datasets.
- Experience designing human-in-the-loop and model-based evaluation workflows, including calibration, error analysis, and validation of evaluation quality.
- Experience integrating evaluation results into CI/CD, release readiness, production monitoring, and governance evidence.
- Advanced expertise in one technical domain and moderate expertise in a different technical domain.
Benefits
- Competitive base salaries and bonus incentives.
- 6% company match on the retirement savings plan.
- Financial coaching and financial well-being support.
- Comprehensive medical, dental, vision, life insurance, and disability benefits.
- Flexible working model with hybrid, onsite, or virtual arrangements depending on the role and business need.
- 20+ weeks of paid parental leave for all parents, including pregnancy, adoption, or surrogacy.
- Access to global onsite wellness centers staffed with nurses and doctors, depending on location.
- Free confidential counseling through the Healthy Minds program.
- Career development and training opportunities.
Tech Stack
Categories
About American Express
American Express builds a global payments network and issues consumer, small-business, and corporate cards, plus merchant acquiring and travel services. Revenue comes from card fees, interest and lending, discount/merchant fees, and network services. Founded in 1850 and headquartered in New York City, it operates the Amex acceptance network worldwide and offers co-branded cards and membership rewards across millions of cardmembers and merchants; ticker NYSE: AXP.