Staff AI Engineer
Modernizing Medicine, Inc.4 months ago
Boca Raton, FL, USAStaff+
Responsibilities
- Define technical direction and contribute to the AI platform roadmap across teams and product domains.
- Lead architecture for large-scale AI/LLM systems, including inference platforms, APIs, compute, cloud, and distributed infrastructure.
- Architect production multi-agent systems covering orchestration, state management, tool integration, enterprise integration, and failure handling.
- Define vector database and RAG architectures using structured RAG, hybrid search, re-ranking, and query routing.
- Set technical direction for MCP adoption, agent runtime infrastructure, frameworks, tooling, and deployment patterns.
- Architect evaluation infrastructure for non-deterministic LLM systems using synthetic golden sets, weighted scoring, confidence intervals, effect-size thresholds, and A/B comparisons.
- Integrate evaluation gates, regression thresholds, tracing, observability, and drift detection into deployment workflows.
- Lead production debugging and incident response, then harden fixes into platform guardrails.
- Drive LLM cost optimization through model routing, caching, batching, token budget management, capacity planning, and provider cost analysis.
- Mentor senior and emerging engineers, lead technical design reviews and ADRs, and influence cross-functional strategy.
Requirements
- Master's or Ph.D. degree in Computer Science, Software Engineering, or a related field.
- 10+ years of professional experience in ML/AI or software engineering, including 4+ years in senior or staff-level roles with production system ownership.
- Demonstrated engineering leadership driving technical strategy and influencing cross-team decisions.
- Expertise in platform and distributed-systems architecture at scale, including model serving, APIs, data platforms, and AI/LLM infrastructure.
- Hands-on experience architecting and operating production agentic AI or LLM systems, including multi-agent workflows, production RAG, tool integration, and MCP integration.
- Deep understanding of embedding models, retrieval algorithms, and vector database internals.
- Strong production debugging, reliability, and incident-response skills.
- Experience evaluating non-deterministic AI systems with statistical methods such as bootstrap confidence intervals and minimum effect-size thresholds.
- Cost awareness for cloud AI/LLM workloads, including capacity planning and cost optimization.
- Proven mentorship of mid-level and senior engineers and strong communication skills for executive and cross-functional audiences.
- Preferred qualifications include experience in healthcare, FinTech, or other regulated industries; building AI/LLM platforms from the ground up; defining AI-assisted development practices with Claude Code; conference talks, published papers, or significant open-source contributions; GPU-accelerated inference and model-serving optimization; and workflow orchestration or streaming architecture experience.
Benefits
- Comprehensive medical, dental, and vision benefits with a company Health Savings Account contribution.
- 401(k) matching contribution of 50% of employee contributions deferred on up to 6% of compensation, with matching contributions fully vested after one year.
- Generous paid time off, paid parental leave, company-paid life and disability benefits, Flexible Spending Account, and Employee Assistance Programs.
- Company-sponsored business resource and special interest groups.
- Professional development opportunities including tuition reimbursement and unlimited LinkedIn Learning access.
- Global presence and in-person collaboration opportunities, with hybrid office-based roles and remote availability for some roles.
- Dog-friendly Boca Raton headquarters with weekly catered breakfast and lunch, treadmill workstations, Zen rooms, and wellness rooms.
Categories
About Modernizing Medicine, Inc.
Modernizing Medicine (ModMed) builds a specialty-specific, cloud-based platform for medical practices that integrates electronic health records, practice management, revenue cycle management, analytics, and patient engagement tools. The company sells its software and services on a subscription and services basis to outpatient specialties such as dermatology, ophthalmology, orthopedics, and gastroenterology. Founded in 2010 and headquartered in Boca Raton, Florida, ModMed is privately held and backed by Clearlake Capital Group.