3 months ago
Base Salary
$167k - $226k/yr
Responsibilities
- Design and evaluate information access and reasoning strategies across RAG, agents, and classic ML, including chunking, embedding models, hybrid search, metadata filtering, and semantic routing.
- Prototype generative AI and agentic workflows that map and reason over compliance objects such as controls, risks, requirements, and evidence.
- Explore classifiers, ranking models, graph/link prediction, calibration, and structured prediction where generative AI is not the best fit.
- Build and maintain evaluation frameworks with golden datasets, automated quality metrics, and regression detection.
- Implement and tune ranking and reranking systems using cross-encoders, LLM-based rerankers, learning-to-rank, and custom scoring functions.
- Run experiments to validate hypotheses and quantify improvements before production rollout.
- Debug failure modes and create error taxonomies across retrieval, reasoning, and generation.
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization.
- Monitor applied research in RAG, agents, LLM evaluation, and relevance modeling and bring relevant innovations into the product.
Requirements
- At least 5 years of experience in applied research, data science, or machine learning focused on NLP, information retrieval, or knowledge systems.
- At least 2 years of hands-on experience building or contributing to production AI/ML systems.
- Strong foundation in dense and sparse retrieval, embedding models, and search relevance.
- Experience building RAG systems, including chunking strategies, vector databases, and retrieval optimization.
- Proficiency with evaluation methodology, including metrics design, golden datasets, A/B testing, and statistical significance.
- Strong Python skills and comfort with notebook-driven research workflows.
- Ability to communicate research findings to engineering teams and translate insights into actionable improvements.
- Experience with compliance, legal, or document-heavy domains is a bonus.
- Publications or contributions in information retrieval, NLP, or RAG evaluation are a bonus.
Benefits
- Hybrid work model with in-office collaboration Tuesday through Thursday in the Bay Area and flexible Mondays and Fridays.
- Stock equity, typically in the form of Restricted Stock Units (RSUs).
- Up to 100% employer-paid medical, dental, and vision premiums for employees and dependents.
- 401(k), company-paid life and disability insurance, tax-advantaged spending accounts, and discounted voluntary benefits.
- Paid parental leave after six months, Kindbody fertility and family-building benefits, and dedicated leave specialists.
- Annual professional and personal development stipends and internal learning opportunities.
- Flexible vacation policy, paid holidays, and additional wellness and work-life balance benefits.
Tech Stack
Categories
About Drata
Drata provides the trust network that enables businesses to operate, scale, and partner with confidence. Powered by AI and designed to operationalize trust, the Drata Agentic Trust Management Platform continuously interprets controls, risk, and assurance signals—reducing repetitive manual work while improving visibility into internal and third-party risk, enabling always-on audit readiness across compliance frameworks, and accelerating security reviews. Purpose-built for enterprise complexity, Drata unifies governance, risk, compliance, and assurance to deliver faster time-to-value, reduce operational overhead, and enable continuous trust for 8,000+ organizations worldwide.