8 days ago
Singapore, SingaporeSenior
Responsibilities
- Design, build, optimize, and maintain scalable batch and streaming data ingestion pipelines using Databricks and Kafka.
- Perform data transformation, cleansing, and validation using PySpark, Python, and SQL.
- Monitor and troubleshoot data workflows, ensuring data quality, reliability, observability, and production readiness.
- Integrate data from files, APIs, databases, and streaming platforms with source-system owners and consuming teams.
- Lead the implementation and operation of knowledge bases, storage lifecycle management, embedding/vectorization, and RAG solutions for GenAI and agentic use cases.
- Provide technical guidance on data platform patterns, reusable components, code quality, deployment readiness, release coordination, and production support.
- Maintain metadata, pipeline documentation, operational runbooks, and traceability practices.
- Conduct incident triage, root-cause analysis, and continuous improvement for assigned data and AI platform components.
- Integrate pipelines with Microsoft Fabric, Databricks, Delta Lake, and other platform components.
- Apply data governance, security, access control, compliance, and operational risk policies.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 5–8 years of experience in data engineering, data platform engineering, or cloud-scale analytics solution delivery.
- Demonstrated ownership of production data pipelines and platform components.
- Experience independently designing, building, optimizing, and operating production-grade batch or streaming pipelines.
- Hands-on experience with Python and SQL for data transformation and validation.
- Familiarity with Apache Spark, especially PySpark, and large-scale data processing.
- Experience implementing knowledge base and RAG solutions for agentic AI use cases.
- Understanding of enterprise data architecture, cloud security, access control, CI/CD, release management, and production operations.
- Ability to lead technical discussions and translate requirements into secure, maintainable platform solutions.
- Strong problem-solving, documentation, communication, and attention-to-detail skills.
Tech Stack
Categories
Data Engineering
About Singtel
Singtel is a Singapore‑headquartered public telecommunications group founded in 1879 that sells mobile, broadband, and enterprise network services to consumers and businesses across Asia-Pacific. It owns Optus in Australia and operates NCS (regional IT services) and Nxera (data centers), and is expanding into AI infrastructure via its RE:AI GPU‑as‑a‑Service. Its revenue comes from connectivity subscriptions and managed services across networks, cloud, cybersecurity, and digital platforms, with equity stakes in regional operators broadening its reach.