3 months ago
Base Salary
$145k - $196k/yr
Responsibilities
- Design and evaluate information access and reasoning strategies across RAG, agents, and classic ML.
- Prototype generative AI and agentic workflows that map and reason over compliance objects such as controls, risks, requirements, and evidence.
- Explore classifiers, ranking models, graph/link prediction, calibration, and structured prediction approaches.
- Build and maintain evaluation frameworks with golden datasets, automated quality metrics, and regression detection.
- Implement and tune ranking and reranking systems, including cross-encoders, LLM-based rerankers, learning-to-rank, and custom scoring functions.
- Run experiments to validate hypotheses and quantify improvements before production rollout.
- Debug failure modes and create error taxonomies across retrieval, reasoning, and generation.
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization.
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling and bring innovations into the product.
Requirements
- At least three years of experience in applied research, data science, or ML focused on NLP, information retrieval, or knowledge systems.
- At least one year of hands-on experience building or contributing to production AI/ML systems.
- Strong foundation in dense and sparse retrieval, embedding models, and search relevance.
- Experience building RAG systems, including chunking strategies, vector databases, and retrieval optimization.
- Proficiency in evaluation methodology, including metrics design, golden dataset creation, A/B testing, and statistical significance.
- Strong Python skills and comfort with notebook-driven research workflows.
- Ability to communicate research findings to engineering teams and translate insights into actionable improvements.
- Experience with compliance, legal, or document-heavy domains is a bonus.
- Publications or contributions in information retrieval, NLP, or RAG evaluation are a bonus.
Benefits
- Hybrid work model with in-office collaboration Tuesday through Thursday in the Bay Area and flexible Mondays and Fridays.
- Stock equity, typically Restricted Stock Units (RSUs).
- Up to 100% employer-paid medical, dental, and vision premiums for employees and dependents, plus wellness benefits and healthcare concierge services.
- 401(k), company-paid life and disability insurance, tax-advantaged spending accounts, and discounted voluntary benefits.
- Paid parental leave after six months of employment, plus Kindbody fertility and family-building benefits.
- Annual professional and personal development stipends and internal learning opportunities.
- Flexible vacation policy and paid holidays.
Tech Stack
Categories
About Drata
Drata provides the trust network that enables businesses to operate, scale, and partner with confidence. Powered by AI and designed to operationalize trust, the Drata Agentic Trust Management Platform continuously interprets controls, risk, and assurance signals—reducing repetitive manual work while improving visibility into internal and third-party risk, enabling always-on audit readiness across compliance frameworks, and accelerating security reviews. Purpose-built for enterprise complexity, Drata unifies governance, risk, compliance, and assurance to deliver faster time-to-value, reduce operational overhead, and enable continuous trust for 8,000+ organizations worldwide.