Senior Software Engineer, Big Data, Tech Enablement
ZipRecruiter3 days ago
Base Salary
$190k - $250k/yr
Responsibilities
- Design, build, deploy, and maintain data ingestion pipelines and datalake systems for web events and downstream applications.
- Develop batch and streaming big-data processing systems and make trade-offs among speed, efficiency, and reliability.
- Build and operate MLOps frameworks that improve the velocity, quality, observability, and reliability of machine-learning development and products.
- Develop data documentation frameworks that support human and automated documentation creation, maintenance, and discovery.
- Build and maintain Sift, an agentic data exploration and analysis tool for data access, collaboration, visualization, and analysis.
- Build and maintain data-platform infrastructure and tools for streaming and batch ingestion, ETL pipelines, data permissions, and data discovery.
Requirements
- 4+ years of professional software development experience, including building and operating data pipelines that feed customer-facing products.
- Hands-on Apache Spark experience writing, tuning, and debugging batch and/or streaming jobs at scale.
- Experience deploying and operating containerized services and batch jobs.
- Strong computer science fundamentals in data structures, algorithms, distributed systems, and testing.
- Preferred: 4+ years of software development focused on data-intensive systems.
- Preferred: working knowledge of AWS services including S3, IAM, EC2, and MSK, plus metrics, logs, and alerts.
- Preferred: experience with datalake technologies including Parquet, object storage, AWS Glue, Apache Iceberg, or Delta Lake.
- Preferred: streaming experience with Kafka, Apache Flink, and/or Spark Structured Streaming.
- Preferred: Apache Airflow experience authoring DAGs, performing backfills, managing dependencies, and supporting scheduled pipelines.
- Preferred: experience with open table-format internals such as schema evolution, partitioning, and compaction.
- Preferred: experience deploying services on Kubernetes with Docker.
- BS, MS, or PhD in Computer Science, Mathematics, Physics, or a related field, or equivalent practical experience.
Benefits
- Hybrid work environment; most US-based positions can be performed remotely unless otherwise noted.
- Competitive compensation and an exceptional benefits package.
- Flexible vacation and paid time off.
- Employer-matched 401(k) plan.
- Additional medical, financial, and other benefits may be provided.
Tech Stack
Categories
Data Engineering
About ZipRecruiter
ZipRecruiter builds an online employment marketplace used by businesses to advertise jobs and by job seekers to find and apply, powered by AI-based matching. The company sells recruiting and job advertising tools to employers through subscriptions and performance-based options, with free tools for candidates. Founded in 2010 and headquartered in Santa Monica, California, ZipRecruiter is a public company listed on the NYSE (ticker: ZIP).