8 days ago
Singapore, SingaporeSenior / Staff+
Responsibilities
- Design, build, optimize, and maintain scalable batch and streaming data ingestion pipelines using Databricks and Kafka.
- Perform data transformation, cleansing, and validation using PySpark and SQL.
- Monitor and troubleshoot data workflows to maintain data quality, reliability, observability, and performance.
- Integrate data from files, APIs, databases, and streaming platforms with source-system owners and consuming teams.
- Provide technical guidance on data platform patterns, reusable components, code quality, deployment readiness, and production support.
- Maintain metadata and pipeline documentation for transparency and traceability.
- Own production readiness for data and AI platform components, including observability, incident triage, root-cause analysis, release coordination, and operational runbooks.
- Integrate pipelines with Microsoft Fabric, Databricks, Delta Lake, and other platform components.
- Build and operate knowledge base and RAG solutions, including storage, lifecycle management, embedding, and vectorization.
- Apply data governance, security, access control, compliance, version control, and CI/CD practices to platform solutions.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 5–8 years of experience in data engineering, data platform engineering, or cloud-scale analytics solution delivery.
- Demonstrated ownership of production-grade batch or streaming pipelines and platform components, including orchestration, observability, error handling, performance tuning, and operational support.
- Hands-on experience with Python and SQL for data transformation and validation.
- Familiarity with Apache Spark, especially PySpark, and large-scale data processing.
- Experience implementing knowledge base and RAG solutions for agentic AI use cases.
- Ability to independently design and operate secure, maintainable production data platform solutions.
- Ability to lead technical discussions with engineers, architects, product owners, source-system teams, and business stakeholders.
- Strong problem-solving, attention to detail, documentation, and communication skills.
- Understanding of enterprise data architecture, cloud security, access control, CI/CD, release management, and production operations.
Tech Stack
Categories
Data Engineering
About Singtel
Singtel is a Singapore‑headquartered public telecommunications group founded in 1879 that sells mobile, broadband, and enterprise network services to consumers and businesses across Asia-Pacific. It owns Optus in Australia and operates NCS (regional IT services) and Nxera (data centers), and is expanding into AI infrastructure via its RE:AI GPU‑as‑a‑Service. Its revenue comes from connectivity subscriptions and managed services across networks, cloud, cybersecurity, and digital platforms, with equity stakes in regional operators broadening its reach.