about 3 hours ago
Remote, United States or San Francisco, CA, USAMid Level
H1B Sponsor
Base Salary
$124k - $255k/yr
Responsibilities
- Design and implement robust data infrastructure in AWS using Spark with Scala.
- Evolve core data pipelines to efficiently scale for massive growth.
- Store data in optimal engines and formats based on performance needs.
- Collaborate with cross-functional teams to design data solutions.
- Design and implement knowledge graphs with Batch Processing and APIs.
- Leverage and optimize AWS resources for scalability.
- Work closely with Data Science and Product teams.
Requirements
- Production data engineering experience.
- Proficiency in Spark and Scala, with experience building data infrastructure.
- Experience delivering significant technical initiatives and large-scale services.
- Experience delivering APIs backed by relationship-heavy datasets.
- Familiarity with data lakes, cloud warehouses, and storage formats.
- Strong proficiency in AWS services.
- Expertise in SQL for data manipulation and extraction.
- Excellent written and verbal communication skills.
- Bachelor's degree in Computer Science or a related field.
- Demonstrated ability to use AI to improve workflow outputs.
- Strong track record of evaluating and verifying AI-assisted work.
- High integrity and ownership in handling sensitive data.
Tech Stack
Categories
AI & MLData Engineering