Member of Technical Staff (Software Engineer, Data Platform)
Perplexity5 months ago
Palo Alto, CA, USA +2 moreSenior / Staff+
H1B sponsor
Base Salary
$220k - $405k/yr
Responsibilities
- Design and operate large-scale batch and streaming data pipelines for product features, AI workloads, analytics, and experimentation.
- Build event-driven ingestion, transformation, and delivery systems alongside batch frameworks for backfills, aggregations, and offline computation.
- Lead data orchestration architecture, including scheduling, dependency management, retries, SLAs, and end-to-end observability.
- Set guarantees for data correctness, freshness, lineage, and recoverability across systems handling scale growth, partial failures, and evolving schemas.
- Build self-serve data platforms that enable engineers, data scientists, and analysts to discover data and operate pipelines.
- Improve developer experience through data modeling, testing, validation, deployment standards, abstractions, and paved paths.
- Drive architectural decisions across storage, compute, orchestration, and data APIs in partnership with product engineering and data science.
- Mentor engineers, review designs, document systems, and raise the technical bar for data infrastructure.
Requirements
- 5+ years of software engineering experience for the Senior track or 8+ years for the Staff track.
- Strong experience building production data infrastructure systems and hands-on experience with batch and/or streaming data processing at scale.
- Deep familiarity with data orchestration systems such as Airflow or Dagster.
- Proficiency in Python and at least one additional backend language such as Go or TypeScript.
- Strong systems thinking around reliability, latency, cost, and complexity tradeoffs.
- Experience supporting ML/AI workflows, training pipelines, or evaluation systems.
- Familiarity with data quality, lineage, observability, and governance tooling.
- Prior ownership of internal platforms used by many teams.
- Applicants are encouraged to apply even if their experience does not match every listed qualification.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkClickHouseDatabricksdbtGoPythonSnowflakeTypeScript
Categories
Data Engineering
About Perplexity
Perplexity builds an AI-powered answer engine for web and mobile and an enterprise product, Perplexity Computer, for AI-driven workflows across tools and apps. The company monetizes through individual subscriptions (Perplexity Pro) and enterprise plans. Founded in 2022 and headquartered in San Francisco, it operates as a privately held company.