8 days ago
Sydney, AustraliaMid Level
Responsibilities
- Manage and monitor distributed systems, storage infrastructure, data processing platforms, and in-house data pipelines.
- Operate and troubleshoot Kafka, HDFS, Dremio, and related large-scale data platforms.
- Drive systems automation and CI/CD to support rapid deployment of hardware and software solutions.
- Collaborate with systems and network engineers, traders, and developers to troubleshoot and support platform queries.
- Evaluate and implement innovative technologies and solutions for the data platform.
Requirements
- At least 2 years of experience managing large-scale, multi-petabyte data infrastructure in a similar role.
- Knowledge of Linux system administration and internals, with demonstrated Linux troubleshooting ability.
- Experience with at least one of Kafka, HDFS, or Kubernetes.
- Working knowledge of Docker, Kubernetes, and Helm.
- Experience with data access technologies such as Dremio and Presto.
- Familiarity with workflow orchestration tools such as Airflow and Prefect.
- Exposure to AWS, GCP, or Azure cloud platforms.
Benefits
- Sydney-based position on a four-person team within the larger APAC Data Team.
- Collaborate with colleagues in Chicago, Mumbai, and Amsterdam.
Tech Stack
Categories
Data EngineeringSite Reliability