7 days ago
Rockville, MD, USA or Reston, VA, USAMid Level
Base Salary
$99k - $168k/yr
Responsibilities
- Design and build LLM pipelines and agents for survey reporting, natural-language survey analysis, multilingual questionnaire translation, and interviewer and supervisor field support.
- Build evaluation harnesses, expert-grounded test sets, automated and human grading processes, regression tests, and ship/no-ship decision workflows.
- Route work across models based on reasoning complexity, volume, cost, latency, and data-residency requirements.
- Track token usage, context limits, prompt caching, batch processing, and inference costs to inform model-selection decisions.
- Connect models to data services and applications built by full-stack developers.
- Use agentic coding tools to produce, review, test, and own production software.
- Monitor production systems, conduct error analysis, and report positive and negative results.
- Document methods and results for public release and transfer tools to partner-country institutions.
Requirements
- Bachelor's degree in computer science, engineering, or a related field, or equivalent years of experience.
- At least 4 years of professional software development experience, including production LLM-powered systems relied on by real users.
- Proficiency in Python or another modern programming language such as JavaScript, TypeScript, or Go.
- Hands-on experience building with frontier-model APIs, tool use, and retrieval-augmented generation.
- Experience designing LLM evaluations, including test-set construction, automated and human grading, regression testing, and ship/no-ship decisions.
- Working knowledge of LLM cost, latency, and model-selection trade-offs.
- Experience building and deploying on a major cloud platform such as AWS, Azure, or Google Cloud.
- Experience using agentic coding tools such as Claude Code, OpenAI Codex, Cursor, Antigravity, or GitHub Copilot agent mode.
- Experience with agent frameworks and AI evaluation tooling.
- U.S. citizenship and authorization to work in the United States are required.
- Preferred qualifications include expert-grounded tasks, structured or tabular data generation, multilingual or low-resource-language NLP, open-weight model deployment, sensitive or regulated data, model governance, global health or international development, and public technical work.
- Strong communication, cross-disciplinary collaboration, candid communication of uncertainty and limitations, and self-directed work are expected.
Benefits
- Remote within the United States with occasional travel.
- Full-time role contingent upon award of program funding.
- Equal opportunity employer with reasonable accommodations available.
Tech Stack
Categories
About ICF
ICF is a global consulting and technology services firm serving government and commercial clients across domains such as energy, health, climate, and digital modernization. It delivers advisory, program implementation, analytics, and software-enabled solutions on platforms like ServiceNow. Founded in 1969 and headquartered in Reston, Virginia, ICF is a public company traded on NASDAQ and works extensively with U.S. federal, state and local agencies as well as regulated industries.
