5 months ago
Base Salary
$180k - $220k/yr
Responsibilities
- Design, build, and ship production ML systems powering autonomous underwriting decisions
- Create feedback loops that convert underwriter behavior into training signals and ongoing model improvement
- Develop confidence scoring and evaluation frameworks to determine when the system should increase autonomy or defer to humans
- Build reliable, auditable, and improvable LLM-based agentic workflows across the underwriting lifecycle
- Work directly with underwriters to extract domain knowledge, validate outputs, and expand the system’s operating domain
- Contribute to observability, monitoring, and guardrail infrastructure for safe AI underwriting
- Own the ML lifecycle from raw data through production and collaborate with underwriters, engineers, and product
Requirements
- 4+ years of industry experience building and shipping end-to-end ML systems, including model deployment platforms such as AWS Sagemaker
- Experience fine-tuning SLMs or LLMs, preferably using RLHF, DPO, or LoRA
- Deep proficiency in Python and modern ML frameworks including PyTorch, HuggingFace, TensorFlow, and OpenAI Gym/Gymnasium or similar
- Production experience with LLMs, including prompt engineering, structured outputs, tool use, evaluation, and cost and latency tradeoffs
- Experience building reliable models with limited labeled data using synthetic data generation, data augmentation, or similar techniques
- Strong evaluation judgment and ability to define measurable improvement criteria
- Comfort working autonomously in ambiguous, high-ownership environments
- Excellent collaboration skills and ability to build trust with non-technical underwriters
- Familiarity with document parsing, information extraction, or NLP for unstructured business documents
- Background in insurance, finance, or another high-stakes structured domain
- Experience with agentic frameworks or multi-step LLM orchestration such as LangChain, LangGraph, or custom systems
- Experience with confidence calibration techniques such as isotonic regression or Platt scaling
- TypeScript proficiency
- Familiarity with data pipelines using SQL, dbt, Spark, or equivalent
- An MS or PhD in a quantitative field such as ML/AI, statistics, mathematics, or physics is preferred
Benefits
- 100% employer contribution to health, dental, and vision coverage
- Fertility benefits and family-building support
- Unlimited paid time off
- Daily lunches, dinners, and snacks
- Offices in San Francisco, New York City, Dallas-Fort Worth, Chicago, and Los Angeles
- Professional development and premium coaching, including leadership development
- Competitive 401(k) plan
- Dog-friendly San Francisco office
Tech Stack
Categories
About Shepherd
Making risk frictionless. Shepherd provides insurance for the builders and operators shaping our physical world.