Senior Data Engineer
Tookitaki Holding PTE LTD3 months ago
Bengaluru, IndiaSenior
Responsibilities
- Build and operate production data platforms across on-premises Cloudera, Cloudera cloud, AWS EMR, and Kubernetes.
- Design batch and streaming pipelines using Spark, Flink, Hive, and related technologies.
- Transform large datasets and design schemas, partitions, storage layouts, compression, retention, and lifecycle strategies.
- Administer Cloudera clusters, including installation, upgrades, configuration, scaling, patching, security, backup, and recovery.
- Troubleshoot distributed data workloads, cluster health, metadata, storage, resource contention, data skew, and infrastructure failures.
- Tune Spark and Flink workloads for memory, CPU, parallelism, shuffle behavior, checkpointing, and recovery.
- Operate AWS services including EMR, S3, IAM, EC2, EKS, CloudWatch, and KMS.
- Deploy and operate data workloads on Kubernetes using operators, Helm, custom resources, and GitOps-based delivery.
- Migrate workloads from physical, virtual-machine, and EC2 environments into AWS and Kubernetes platforms.
- Build monitoring, alerting, capacity-management processes, operational runbooks, and production automation.
- Automate platform provisioning and configuration using Terraform, Ansible, scripting, and CI/CD.
- Support production incidents and collaborate with data engineering, infrastructure, security, and application teams.
Requirements
- Strong hands-on production experience administering Cloudera platforms, including both on-premises and cloud deployments.
- Deep knowledge of Hadoop, HDFS, Hive, YARN, Spark, and distributed-data processing.
- Experience building or operating production workloads with Apache Flink.
- Experience with partitioning, shuffling, serialization, checkpointing, failure recovery, schema evolution, and large-scale data transformations.
- Practical knowledge of Parquet, Avro, JSON, CSV, compression formats, and serialization tradeoffs.
- Strong AWS experience, particularly with EMR, S3, EC2, EKS, IAM, CloudWatch, and KMS.
- Strong Kubernetes experience with operators, controllers, Helm, scheduling, storage, networking, and workload troubleshooting.
- Experience migrating data platforms from on-premises or VM-based environments into AWS and Kubernetes.
- Ability to troubleshoot Linux, JVM, networking, storage, DNS, certificates, and resource-management issues.
- Experience with observability platforms and correlating infrastructure symptoms with data-pipeline failures.
- Ability to automate operational work using Python, Bash, Terraform, Ansible, or equivalent tools.
- Strong judgment regarding production changes, data integrity, access control, rollback, and recovery.
- Relevant certifications are useful, including Cloudera Certified Professional: Data Engineer, Cloudera Administrator, AWS data or architecture certifications, CKA, or CKAD.
Tech Stack
AnsibleApache FlinkApache HadoopApache HiveApache SparkAWSBashHelmKubernetesLinuxPythonTerraformYarn
Categories
Data EngineeringDevOps
About Tookitaki Holding PTE LTD
Tookitaki builds AI-driven anti-financial-crime software for banks and fintechs, covering AML compliance, customer screening, risk scoring, transaction monitoring, and real-time fraud detection. Its FinCense platform is delivered to enterprise clients across cloud and on‑premises environments. Founded in 2015 and headquartered in Singapore, the privately held company is part of Thunes and raised a Series B round in 2022.