Senior AI Observability & Telemetry Engineer
BitDeer Technologies Group6 hours ago
Singapore, SingaporeSenior
Responsibilities
- Architect and scale high-cardinality telemetry infrastructure using highly available time-series databases.
- Integrate NVIDIA DCGM, network switch telemetry, IPMI, and Redfish exporters into the Kubernetes observability stack.
- Build eBPF-based diagnostic tools for network congestion, kernel-level I/O latency, and distributed training bottlenecks.
- Develop dashboards and alerting pipelines that proactively cordon degraded hardware.
- Design metric pipelines for multi-tenant consumption billing based on real-time GPU and network utilization.
- Create observability standards for AI-native workloads in collaboration with GPU Systems and Scheduling teams.
- Lead technical design reviews and mentor team members on telemetry collection and analysis best practices.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or a related field.
- 6+ years of software or site reliability engineering experience.
- Deep hands-on expertise with the Prometheus and OpenTelemetry ecosystems.
- Advanced proficiency in Go and experience writing custom Kubernetes metric exporters and operators.
- Hands-on experience with eBPF and BCC, plus deep Linux performance-tuning experience.
- Familiarity with AI hardware metrics, GPU power states, SM utilization, memory bandwidth, and high-performance network telemetry.
- Experience operating, debugging, and scaling large-scale telemetry stacks in HPC or cloud environments.
- Strong technical leadership, architectural influence, cross-functional collaboration, and communication skills.
- Experience in high-velocity, high-growth engineering environments is strongly preferred.
Benefits
- Inclusive and diverse work environment with open workspaces and startup spirit.
- Opportunity to contribute directly to digital asset and AI cloud projects and network with industry pioneers.
- Personal accountability, autonomy, growth, and learning opportunities.
- Training, mentoring, welfare benefits, and developmental opportunities.
Tech Stack
Categories
About BitDeer Technologies Group
Bitdeer Technologies Group (NASDAQ: BTDR) builds and operates Bitcoin‑mining infrastructure and AI/HPC cloud services for miners and enterprise compute customers. The Singapore‑headquartered public company, founded in 2021, designs ASIC chips and manufactures mining rigs, and runs data centers across the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia with a diversified 3 GW energy portfolio.