AI Engineer, AI & Applications
Firmus Technologies2 days ago
Sydney, AustraliaMid Level
Responsibilities
- Design, develop, and maintain end-to-end MLOps workflows across development, staging, and production environments.
- Own MLflow Model Registry workflows, including stage and alias strategies, approvals, access control, rollback readiness, and release-pipeline integration.
- Establish model and dataset reproducibility, lineage, artifact traceability, and repeatable evaluation processes.
- Develop automated production quality gates covering accuracy, latency, cost, safety, and reliability.
- Define SLOs and SLIs for training jobs, inference services, RAG retrieval, availability, and cost.
- Build monitoring dashboards for model performance, data drift, and operational health, integrating alerts with PagerDuty and Slack.
- Create on-call runbooks, triage procedures, and incident-response processes for AI services.
- Instrument AI services with request traces, GPU metrics, retrieval performance, and communication diagnostics.
- Integrate benchmarking, RAGAS, and LLM-as-judge evaluation frameworks into CI/CD and release workflows.
- Partner with ML engineers, data scientists, platform teams, application engineers, and operations teams on production reliability and L2/L3 incident response.
Requirements
- Bachelor’s degree or higher in Software Engineering, Computer Science, Information Technology, or a closely related field.
- At least 3 years of experience in MLOps, ML platform, software engineering, or production engineering roles.
- Hands-on ownership of production machine-learning delivery pipelines.
- Deep understanding of model versioning, promotion strategies, evaluation automation, governance, deployment, monitoring, and drift detection.
- Hands-on experience with MLflow Model Registry workflows, including stages or aliases, approvals, traceability, and release-pipeline integration.
- Experience designing and operationalising model evaluation systems with metrics standards, orchestration, logging, and reproducibility.
- Strong software engineering and production fundamentals covering metrics, logs, traces, alerting, incident response, and reliability practices.
- Familiarity with CI/CD pipelines, model packaging, deployment automation, and software testing.
- Understanding of distributed systems, resource management, and failure modes in training and inference environments.
- Ability to define SLOs, error budgets, monitoring practices, and postmortem-driven improvements.
- Ability to communicate complex ML and systems issues to technical and non-technical stakeholders.
Benefits
- Full-time position based in Sydney, Australia.
- Opportunity to work on sustainable AI infrastructure and Firmus AI Factory.
- Inclusive workplace encouraging applications from candidates of all backgrounds.
Tech Stack
MLflow
Categories
About Firmus Technologies
Firmus Technologies builds energy‑efficient AI infrastructure, developing liquid‑cooled “AI Factory” data centers and operating a large‑scale GPU cloud for model training. The company sells capacity and services to developers, enterprises, education, and government customers, with a focus on energy and cost efficiency across Asia‑Pacific. Founded in 2019 in Australia, Firmus is privately held and headquartered in St Leonards, Tasmania.