Staff Software Engineer, Data Infrastructure

3 days ago

Washington, DC, USAStaff+

Base Salary

$200k - $275k/yr

Responsibilities

Design and operate a high-throughput, real-time data integration platform across diverse customer environments.
Architect a scalable open table format layer for reliable data storage at petabyte scale.
Build and optimize distributed data processing pipelines for batch and streaming workloads.
Drive performance, reliability, and cost efficiency across the data infrastructure stack.
Define data contracts, schemas, and integration patterns with platform and product engineering teams.
Establish best practices, tooling, and patterns that improve data infrastructure quality across the organization.

8+ years of experience architecting and operating large-scale data infrastructure systems in production.
Deep expertise with open table formats, particularly Apache Iceberg, including schema evolution, partitioning, compaction, and time travel.
Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale.
Strong real-time data integration and stream-processing experience with Apache Kafka, Apache Flink, or equivalent technologies.
Experience with data pipeline orchestration using Airflow or similar tools.
Strong software engineering fundamentals in Python and/or Scala with a record of writing production-quality code.
Extensive experience with AWS or comparable cloud platforms, including S3-based data lake architectures.
Experience with Kubernetes and containerized deployment of data workloads.
Degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
Located in San Francisco, New York, or Washington DC and open to working in office.

Benefits, equity, and bonus may be available as applicable.
The role is based in San Francisco, New York, or Washington DC and requires openness to working in office.
Peregrine is committed to an inclusive environment and is an equal opportunity employer.

501-1,000 employees