Software Engineer, Big Data, Tech Enablement
ZipRecruiter1 day ago
Remote, United StatesMid Level
H1B sponsor
Base Salary
$135k - $185k/yr
Responsibilities
- Design, build, deploy, and maintain data ingestion pipelines and datalake systems for web events and downstream applications.
- Develop batch and streaming big data processing systems and balance speed, efficiency, and reliability for large evolving datasets.
- Build and operate MLOps frameworks that improve the velocity, quality, observability, and reliability of machine learning development and products.
- Create frameworks and tooling that improve data documentation comprehensiveness, quality, maintenance, and discoverability.
- Build and maintain Sift, an agentic data exploration and analysis tool for data access and decision-making.
- Build and maintain data platforms supporting streaming and batch ingestion, ETL pipelines, data permissions, data discovery, and user-facing products.
Requirements
- At least 2 years of professional software development experience, including building and operating data pipelines that feed customer-facing products.
- Hands-on Apache Spark experience writing, tuning, and debugging batch or streaming jobs at scale.
- Experience deploying and operating containerized services and batch jobs.
- Strong computer science fundamentals in data structures, algorithms, distributed systems, and testing.
- Preferred: 4 or more years of professional software development experience focused on data-intensive systems.
- Preferred: knowledge of AWS services including S3, IAM, EC2, and MSK, plus job instrumentation with metrics, logs, and alerts.
- Preferred: experience with Parquet on object storage, AWS Glue, Apache Iceberg or Delta Lake, Kafka, Apache Flink, Spark Structured Streaming, and Apache Airflow.
- Preferred: experience deploying services on Kubernetes with Docker and building data platforms for user-facing products.
- BS, MS, or PhD in Computer Science, Mathematics, Physics, or a related field, or equivalent practical experience.
Benefits
- Hybrid work environment; most US-based positions can be performed remotely unless otherwise noted.
- Competitive compensation and an exceptional benefits package.
- Flexible vacation and paid time off.
- Employer-matched 401(k) plan.
Tech Stack
Categories
Data Engineering
About ZipRecruiter
ZipRecruiter builds an online employment marketplace used by businesses to advertise jobs and by job seekers to find and apply, powered by AI-based matching. The company sells recruiting and job advertising tools to employers through subscriptions and performance-based options, with free tools for candidates. Founded in 2010 and headquartered in Santa Monica, California, ZipRecruiter is a public company listed on the NYSE (ticker: ZIP).