Member of Technical Staff (Software Engineer, Data Platform)
Perplexityabout 3 hours ago
Palo Alto, CA, USA +2 moreSenior / Staff+
H1B Sponsor
Base Salary
$220k - $405k/yr
Responsibilities
- Design and operate large-scale batch and streaming data pipelines.
- Build event-driven and streaming systems for real-time data processing.
- Lead the architecture of data orchestration using tools like Airflow or Dagster.
- Set and enforce guarantees for data correctness, freshness, and lineage.
- Build self-serve data platforms for engineers and data scientists.
- Improve developer experience through better abstractions and standards.
- Drive architectural decisions across storage, compute, and data APIs.
- Mentor engineers and raise the technical bar for data infrastructure.
Requirements
- 5+ years of software engineering experience for Senior role or 8+ years for Staff role.
- Strong experience building production data infrastructure systems.
- Hands-on experience with batch and/or streaming data processing at scale.
- Deep familiarity with data orchestration systems like Airflow or Dagster.
- Proficiency in Python and at least one additional backend language.
- Strong systems thinking around reliability, latency, and complexity tradeoffs.
- Experience supporting ML/AI workflows and training pipelines.
- Familiarity with data quality, lineage, and observability tooling.
- Prior ownership of internal platforms used by multiple teams.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkClickHouseDatabricksdbtGoPythonSnowflakeTypeScript