1 day ago
Base Salary
$272k - $431k/yr
Responsibilities
- Architect and scale a unified data acquisition platform covering source discovery, URL indexing, crawl frontiers, ranking, scheduling, fetching, and ingestion.
- Design high-throughput frontier storage and queueing systems with efficient lookup, deduplication, discovery feedback loops, and quality-aware scheduling.
- Build reliable crawl execution and extensible pipelines for open-web, licensed, partner, text, PDF, code-repository, and multimodal content.
- Implement provenance, privacy, licensing, publisher opt-out, and policy-enforcement controls that remain auditable downstream.
- Own production readiness through capacity planning, observability, fault tolerance, recovery, service objectives, and cost efficiency.
- Evaluate crawls and datasets through reproducible pilots and releases tied to controlled training experiments.
- Lead architecture reviews, critical implementation work, cross-team dependency coordination, technical tradeoffs, mentoring, and team building.
Requirements
- Bachelor’s, master’s, or doctoral degree in Computer Science, Computer Engineering, or a related technical field, or equivalent experience.
- 15+ years of relevant software engineering experience designing and operating large-scale distributed systems and setting technical direction across multiple teams.
- Deep experience with web crawling or related discovery, indexing, ingestion, scheduling, stateful services, storage, and high-throughput data processing infrastructure.
- Strong hands-on system design, implementation, performance analysis, and production debugging skills.
- Experience building batch or streaming pipelines and handling consistency, retries, duplicate processing, backpressure, recovery, and operational reliability.
- Experience delivering data platforms for research, machine learning, search, or other demanding technical users.
- Strong technical communication and influence across engineering, product, research, and governance teams.
- Preferred experience includes web-wide crawl platforms, foundation-model pretraining data, quality-aware crawl ranking, publisher consent and licensing controls, provenance systems, browser-based or agentic collection, sandboxed environments, reusable partner integrations, and low-latency multimodal data feeds.
Benefits
- Eligible for equity and benefits.
- Base salary range of 272,000 USD to 431,250 USD, determined by location, experience, and comparable employee pay.
- Applications accepted at least until October 30, 2026.
Categories
BackendData Engineering
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.