Software Engineer
Spellbrushover 1 year ago
Tokyo, Japan or San Francisco, CA, USAMid Level / Senior
H1B Sponsor
Responsibilities
- Define processes and infrastructure for data transformation and availability.
- Partner with AI researchers to understand their data needs.
- Design, build, and monitor scalable data pipelines.
- Work with various data sources and tools to manage large datasets.
- Communicate effectively about deliverables, timelines, and tradeoffs.
Requirements
- 3+ years of experience as a software engineer with a generalist skillset.
- Experience with distributed systems and big data tools like Ray, Spark, or Airflow.
- Hands-on experience with cloud services such as AWS, GCP, or Azure.
- Proficient in SQL and comfortable with scripting in Typescript, Python, or Bash.
- Ability to design and build scalable and reliable data pipelines.
Benefits
- Opportunity to work with leading AI researchers.
- Collaborative and fast-paced team environment.
- Visa sponsorship available for international candidates.
Tech Stack
Amazon RedshiftApache AirflowApache HiveApache SparkAWSAzureBashGoogle BigQueryGoogle Cloud PlatformPythonSnowflakeSQLTypeScript