about 3 hours ago
Remote, WorldwideMid Level / Senior
Responsibilities
- Design and execute fine-tuning strategies for structured generation and editing workflows.
- Build supervised datasets from successful generations, retries, failures, and user edits.
- Develop measurable benchmarks for generation quality, correctness, and edit preservation.
- Experiment with open-source models such as Llama, Qwen, and Mistral.
- Implement LoRA, QLoRA, supervised fine-tuning, and synthetic data approaches.
- Build automated pipelines for collecting, cleaning, and evaluating production data.
- Use validation systems and runtime analysis as structured feedback signals for models.
- Improve retry, repair, and self-correction workflows for generation pipelines.
- Collaborate cross-functionally with engineering, design, and product teams.
Requirements
- Strong experience building with LLMs or structured generation systems in production.
- Hands-on experience fine-tuning or adapting open-source language models.
- Strong Python engineering skills.
- Experience building evaluation systems, ML experimentation workflows, or data pipelines.
- Strong understanding of prompt engineering and model failure analysis.
- Ability to define measurable evaluation criteria.
- Comfort debugging systems spanning model outputs and validation systems.
- Strong communication and collaboration skills.