Senior Software Engineer, Big Data, Tech Enablement
ZipRecruiter3 days ago
Base Salary
$190k - $250k/yr
Responsibilities
- Design, build, deploy, and maintain data ingestion pipelines and systems for web events, data lakes, and downstream event-stream consumers.
- Develop batch and streaming big data processing systems for large and evolving datasets.
- Build and maintain MLOps frameworks that improve the velocity, quality, observability, and reliability of ML development and ML-powered products.
- Develop data documentation frameworks that enable humans and automated processes to create, maintain, and discover documentation.
- Build and maintain Sift, an agentic data exploration and analysis tool for data discovery, querying, visualization, and analysis.
- Build and maintain data platforms supporting streaming and batch ingestion, ETL pipelines, data permissions, and data discovery.
Requirements
- At least 4 years of professional software development experience, including building and operating data pipelines that feed customer-facing products.
- Hands-on Apache Spark experience writing, tuning, and debugging batch or streaming jobs at scale.
- Experience deploying and operating containerized services and batch jobs.
- Strong knowledge of data structures, algorithms, distributed systems, and testing.
- Experience with data-intensive systems, data lakes, streaming technologies, workflow orchestration, and data platform infrastructure is preferred.
- Experience with AWS services such as S3, IAM, EC2, and MSK, or equivalent technologies, is preferred.
- Experience with Parquet, object storage, AWS Glue or another data catalog, Apache Iceberg or Delta Lake, Kafka, Apache Flink, Spark Structured Streaming, Apache Airflow, Kubernetes, and Docker is preferred.
- Bachelor’s, master’s, or doctoral degree in Computer Science, Mathematics, Physics, or a related field, or equivalent practical experience.
Benefits
- Hybrid work environment; most US-based positions may also be performed remotely unless an exception is specified.
- Competitive compensation and an exceptional benefits package.
- Flexible vacation and paid time off.
- Employer-matched 401(k) plan.
- Full-time position with potential additional equity, bonus, commission, or other compensation as part of the total compensation package.
Tech Stack
Categories
Data Engineering
About ZipRecruiter
ZipRecruiter builds an online employment marketplace used by businesses to advertise jobs and by job seekers to find and apply, powered by AI-based matching. The company sells recruiting and job advertising tools to employers through subscriptions and performance-based options, with free tools for candidates. Founded in 2010 and headquartered in Santa Monica, California, ZipRecruiter is a public company listed on the NYSE (ticker: ZIP).