9 hours ago
Base Salary
$157k - $252k/yr
Responsibilities
- Design and evaluate retrieval and reasoning strategies across RAG, agents, and classic ML, including chunking, embeddings, hybrid search, metadata filtering, and semantic routing.
- Prototype agentic GenAI workflows that map and reason over compliance controls, risks, requirements, and evidence.
- Explore classifiers, ranking models, graph/link prediction, calibration, and structured prediction approaches where GenAI is not the best fit.
- Build and maintain golden datasets, automated quality metrics, regression detection, and other evaluation frameworks.
- Implement and tune ranking and reranking systems using cross-encoders, LLM-based rerankers, learning-to-rank, and custom scoring functions.
- Run experiments, quantify improvements, debug failure modes, and create error taxonomies across retrieval, reasoning, and generation.
- Collaborate with AI and software engineers to productionize validated approaches and stay current with applied research in RAG, agents, LLM evaluation, and relevance modeling.
Requirements
- At least 6 years of experience in applied research, data science, or ML focused on NLP, information retrieval, or knowledge systems.
- At least 2 years of hands-on experience building or contributing to production AI/ML systems.
- Strong foundations in dense and sparse retrieval, embedding models, and search relevance.
- Experience building RAG systems, including chunking strategies, vector databases, and retrieval optimization.
- Proficiency in evaluation methodology, including metric design, golden dataset creation, A/B testing, and statistical significance.
- Strong Python skills and comfort with notebook-driven research workflows.
- Ability to communicate research findings to engineering teams and translate insights into actionable improvements.
- Experience in compliance, legal, or document-heavy domains is a bonus.
- Publications or contributions in information retrieval, NLP, or RAG evaluation are a bonus.
Benefits
- Stock equity and an opportunity to share in company success.
- Up to 100% employer-paid medical, dental, and vision premiums for employees and dependents, plus wellness benefits and healthcare concierge services.
- 401(k), company-paid life and disability insurance, tax-advantaged spending accounts, and discounted voluntary benefits.
- Paid parental leave after six months of employment, plus Kindbody fertility and family-building benefits and leave specialists.
- Annual professional and personal development stipends and access to internal learning opportunities.
- Flexible vacation policy, paid holidays, and other time-off benefits.
Tech Stack
Categories
About Drata
Drata builds a trust management platform that automates security compliance, risk, and audit readiness across frameworks like SOC 2, ISO 27001, HIPAA, and PCI DSS. Delivered as SaaS, it connects to cloud and business systems to monitor controls, manage GRC workflows, and streamline security reviews for thousands of organizations. Founded in 2020 and privately held, Drata is headquartered in San Francisco, California.