Technical Solutions Architect, Evals & Fine-Tuning
Innodata Inc.2 months ago
Remote, United StatesSenior
Base Salary
$140k - $160k/yr
Responsibilities
- Lead technical discovery with foundation model labs, frontier AI teams, and enterprise customers.
- Design end-to-end post-training solutions covering SFT, preference optimization, evaluation, human feedback, red teaming, and multimodal evaluation.
- Architect engagements using Innodata’s GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench, and global SME workforce.
- Author technical proposals, statements of work, solution diagrams, and pricing models with sales, delivery, and finance.
- Run technical workshops, proofs of concept, and pilot designs to demonstrate value and de-risk larger programs.
- Advise customers during delivery and coordinate with research, engineering, data science, and program teams.
- Provide customer insights to Innodata’s R&D and product roadmap and represent the company externally.
Requirements
- At least 7 years of experience in applied machine learning, ML engineering, ML research, or technical solutions, including at least 2 years focused on LLM evaluation or post-training.
- Hands-on experience fine-tuning LLMs, including SFT and preferably RLHF, DPO, or KTO, plus experience designing supporting data pipelines.
- Deep knowledge of LLM evaluation methodology, benchmark construction, LLM-as-judge failure modes, inter-annotator agreement, and human evaluation workflows.
- Strong Python fluency and experience with Hugging Face, PyTorch, vLLM, and evaluation frameworks such as lm-evaluation-harness or lighteval.
- Excellent technical communication with both research-scientist and executive audiences.
- Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or a related field, or equivalent demonstrated experience.