1 day ago
Foster City, CA, USA or Seattle, WA, USA
Mid Level / Senior
H1B Sponsor
Base Salary
$175k - $230k/yr
Responsibilities
- Develop features for the schedule-based processing framework using Airflow, AWS EMR, and DuckDB.
- Improve stability, performance, and scalability of data ingestion and processing platforms.
- Collaborate with cross-functional teams to gather requirements and design architectures.
- Translate user pain points into technical solutions and roadmap items with senior engineers.
- Build monitoring and alerting tools to enhance system observability.
Requirements
- Bachelor's or Master's in Computer Science or related fields with 4+ years of industry experience in software engineering.
- Strong background in Python for large-scale data processing.
- Familiarity with large-scale data processing systems like Spark, Trino, and DuckDB.
- Experience using cloud services such as AWS, GCP, or Azure.
- Strong experience in troubleshooting data pipeline failures and optimizing performance and cost efficiency.
Tech Stack
Apache AirflowApache SparkAWSAzureGoogle Cloud PlatformPython
Categories
BackendData EngineeringData Science