Perplexity

Member of Technical Staff (Index Engineer, Search Crawler)

Perplexity
Apply
1 day ago
London, United KingdomSenior

Responsibilities

  • Design, implement, and operate large-scale indexing systems and data pipelines for search infrastructure.
  • Develop indexing strategies that balance performance, freshness, and resource efficiency.
  • Build storage formats, compaction, and update mechanisms for current and accessible data.
  • Improve pipeline reliability and predictability under high-throughput conditions.
  • Build well-tested components with clear responsibilities and flexible interface contracts.
  • Define observability for offline and nearline pipelines using structured logs, metrics, and data quality signals.
  • Monitor throughput, resource usage, and cost, and drive optimizations.
  • Partner with retrieval, ranking, and ML teams to meet downstream indexing requirements.
  • Enable safe experimentation through controlled rollouts and clearly defined quality signals.

Requirements

  • At least 5 years of experience building production backend or data infrastructure systems.
  • Strong programming experience in Go, C++, or Rust.
  • Experience with high-throughput data processing systems and very large datasets.
  • Experience building or operating databases, storage systems, data planes, or indexing pipelines.
  • Strong understanding of distributed systems, fault tolerance, consistency, and scalability.
  • Experience running production systems and handling operational incidents.
  • Ability to reason about end-to-end data flows with a systems-thinking mindset.

Tech Stack

Categories

BackendData Engineering
Perplexity

About Perplexity

1,001-5,000 employees

Perplexity builds an AI-powered answer engine for web and mobile and an enterprise product, Perplexity Computer, for AI-driven workflows across tools and apps. The company monetizes through individual subscriptions (Perplexity Pro) and enterprise plans. Founded in 2022 and headquartered in San Francisco, it operates as a privately held company.

Contact me