about 5 hours ago
Base Salary
$285k - $335k/yr
Responsibilities
- Develop a unified observability plane for diagnosing and recovering jobs quickly.
- Create a system where tens of thousands of accelerators operate as a single programmable entity.
- Implement closed-loop autonomy to automatically diagnose and remediate issues.
- Maximize compute goodput by optimizing scheduling and maintenance decisions.
- Predict hardware failures ahead of time to prevent job stalls.
- Detect stragglers and silent failures to ensure job efficiency.
- Schedule workloads based on real-time energy availability and cost.
- Simulate system behaviors to ensure safe autonomy before production deployment.
- Design an agent that autonomously proposes and executes fixes for slow jobs.
- Establish a zero-trust, auditable multi-tenancy architecture.
- Ensure new hardware passes automated burn-in tests before customer use.
Requirements
- 10+ years of experience in building infrastructure-layer systems at scale.
- Deep knowledge of distributed systems design and closed-loop automation.
- Hands-on experience with GPU/HPC infrastructure and telemetry.
- Proven track record in designing large-scale observability platforms.
- Ability to operate in ambiguity and define new system architectures.
- Strong software engineering skills in systems programming languages.
- Experience with ML/statistical methods for operational telemetry is a plus.
- Familiarity with zero-trust or policy-based multi-tenancy architectures is a plus.
Benefits
- Competitive compensation and equity packages.
- Restricted Stock Units.
- Paid time off and holidays.
- Comprehensive health, dental, and vision insurance.
- Employer contributions to HSA accounts.
- Paid parental leave and life insurance.
- Professional development and tuition reimbursement.
- Mental health and wellness support.
- Commuter benefits and cell phone stipend.
- 401(k) retirement plan with company match.
- Volunteer time off and global travel insurance.
- Daily meals allowance and additional location-specific perks.