Staff Platform Engineer - Big Data Platform Management
Centre for Strategic Infocomm Technologies5 days ago
Singapore, SingaporeStaff+
Responsibilities
- Design, implement, and manage the Big Data Infrastructure platform, including Hadoop, Spark, NoSQL databases, and data lakes.
- Monitor, troubleshoot, and optimize Big Data clusters, including job scheduling and resource allocation.
- Design and implement storage solutions using HDFS, S3, and other object storage systems.
- Maintain platform security and governance through access controls, encryption, authentication, authorization, and security audits.
- Collaborate with data engineers and data scientists to optimize application performance and provide platform guidance.
- Plan and manage Big Data resource capacity, including forecasting, procurement, and deployment of hardware and software.
- Support search workloads involving multi-terabyte datasets and billions of records.
- Lead engineering teams.
Requirements
- Background in Computer Science, Engineering, or a related field.
- At least 8 years of experience managing Big Data platforms.
- Experience with Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka, and Airflow.
- Expertise in Big Data Engineering, including streaming and batch pipelines, data warehouses, and AI/ML infrastructure.
- Experience with S3, NFS, SMB, and HDFS storage protocols.
- Knowledge of SSL/TLS encryption, encryption at rest, OIDC/OAuth2, authentication, and authorization.
- Understanding of firewall configuration, DNS management, routing, and subnetting.
- Experience with Bash, Python, Scala, and Java scripting or programming.
- Proficiency with Kubernetes orchestration and its ecosystem.
- Experience with agile development methodologies and version control systems such as Git.
- Experience leading engineering teams.
- Big Data certifications such as Hadoop Certified Developer or Spark Certified Developer are nice to have.
- Experience managing physical and virtual infrastructure is nice to have.
- Knowledge of deploying and scaling LLM-based and agentic AI/ML applications is nice to have.
Benefits
- Purposeful and meaningful work supporting Singapore’s national security interests.
- Opportunity to work with experienced engineers and modern technologies.
- Engineering culture focused on work-life balance, innovation, and operational excellence.
- Professional growth and collaborative team environment.
- Only Singapore Citizens will be considered.
Tech Stack
Categories
Data EngineeringDevOps
About Centre for Strategic Infocomm Technologies
The Centre for Strategic Infocomm Technologies (CSIT) is a Singapore government agency that develops cybersecurity, data analytics, and software systems to support national security and public-sector operations. It conducts applied research, builds AI and cyber‑defence tools in a self‑managed secure cloud, and delivers enterprise applications for internal users. Established in 2003 and based in Singapore, CSIT is part of the Defence Technology Community.