Senior Software Engineer, Data Infrastructure
Arena Intelligence, Inc.9 months ago
Responsibilities
- Design and build robust data pipelines to ingest, process, and transform user vote data into features for model performance evaluation.
- Collaborate with researchers and product leadership to identify product goals and required data.
- Design and implement dashboards and reports for the public, model providers, and researchers.
- Ensure the integrity, quality, and reliability of data pipelines.
- Scale data infrastructure to support increasing data volumes and evolving analytical needs.
Requirements
- 5+ years of software engineering experience with a dedicated focus on data engineering and big data technologies.
- Proficiency in SQL and at least one data-analysis programming language, with Python preferred and Scala or R also accepted.
- Hands-on experience with data processing and pipeline frameworks such as Apache Spark or Ray Data.
- Experience with at least one big data analytics platform, such as Databricks or Snowflake.
- Demonstrated experience designing, implementing, optimizing, and debugging production data pipelines.
- Prior work with data analytics or data lake platforms is preferred.
- Experience with advanced data analysis tools such as Delta Lake and streaming tables is preferred.
- Exposure to machine learning is a plus.
Benefits
- Competitive compensation and equity aligned to the candidate’s market, with the base salary range depending on permanent work location.
- Comprehensive medical, dental, vision, and additional health and wellness support programs.
- Opportunity to work on cutting-edge AI with a small, mission-driven team.
- Culture emphasizing transparency, trust, and community impact.
Tech Stack
Categories
Data Engineering
About Arena Intelligence, Inc.
Created by researchers from UC Berkeley, Arena (formerly LMArena) is a community-powered platform to measure and advance the frontier of AI for real-world use. Tens of millions of builders, researchers, and creative professionals come to Arena to use frontier models and give feedback on their responses, shaping a public leaderboard grounded in real-world use.