8 months ago
Base Salary
$180k - $350k/yr
Responsibilities
- Design a lakehouse architecture that handles over 100 PB of web crawl data.
- Build streaming pipelines that process billions of documents daily for real-time indexing.
- Architect the data layer for embedding training infrastructure on Ray.
- Scale ClickHouse deployment to manage analytical queries across petabytes of search logs.
Requirements
- Deep understanding of lakehouse architectures like Delta Lake, Iceberg, and Hudi.
- Experience in building and operating large-scale distributed data processing pipelines.
- Hands-on experience with streaming data systems such as Kafka or Flink.
- Familiarity with Ray, Spark, or ClickHouse at production scale.
- Focus on reliability to build systems that minimize operational issues.
Benefits
- Premium healthcare benefits including medical, dental, and vision.
- Fertility benefits offered to all employees.
- Monthly wellness stipend provided.
Tech Stack
Categories
Data Engineering
About Exa
Exa is an applied AI research lab organizing human knowledge, starting with the web. Exa Search is the highest quality search API at every latency. It accepts natural language queries and produces token-efficient results with citations. Exa Agent is a high-compute web agent that synthesizes information from across the web to handle the most complex deep research, list-building, and enrichment workflows. These products power Cursor, Cognition, Hubspot, OpenRouter, Monday.com, and agents built by over 400,000 developers. We are a worldwide team of engineers and researchers. We're hiring: exa.ai/careers