AI Engineer, Model Quality and Performance
Cerebras Systems4 months ago
Responsibilities
- Design evaluation suites using AI agents for advanced, basic, long-context, and customer-specific use cases.
- Build custom evaluations by mining customer workload trajectories and synthesizing representative evaluation sets.
- Automate end-to-end evaluation execution and release qualification using Docker, Git, CI, and AI-driven pipelines.
- Forecast and benchmark model performance on Cerebras for customer workloads, including production-speed modeling.
- Build product-quality tooling that presents model quality and performance data in a unified view.
Requirements
- Experience building and shipping real AI-agent systems using Claude or an equivalent technology.
- Strong mathematics and statistics background.
- Comfort with Docker, Git, and standard automation tooling.
- Experience designing usable tools adopted by non-engineers.
- Preferred: performance tuning on custom silicon, GPUs, or FPGAs.
- Preferred: evaluation design for agentic, coding, long-context, or multimodal use cases.
- Preferred: familiarity with EvalScope, lm-eval-harness, or similar open-source evaluation frameworks.
Benefits
- Opportunity to build an AI platform beyond GPU constraints and work on a high-speed AI supercomputer.
- Opportunities to publish and open-source AI research.
- Job stability with startup vitality and a simple, non-corporate work culture.
- Equal-opportunity environment focused on inclusion, learning, growth, and support.
Categories
About Cerebras Systems
Cerebras Systems designs and sells AI compute systems built around its wafer-scale WSE-3 processor, delivered as the CS-3 appliance and via the Cerebras Cloud. It targets enterprises, model labs, and government users needing fast training and inference, and offers on‑prem and cloud deployments. Privately held and headquartered in Sunnyvale, California, the company announced a multi-year partnership with OpenAI to deploy large-scale inference capacity.