Visa

Staff Data Platform Engineer (AWS, Databricks, Azure, IaC, Solution Architect, Devops, Java/Python, 6 - 10 Years)

Visa
Apply
1 month ago
Bengaluru, IndiaStaff+

Responsibilities

  • Design, develop, and maintain scalable, secure, and reliable cloud-based data platform services across AWS, Azure, and Databricks.
  • Build reusable platform capabilities, self-service onboarding solutions, engineering frameworks, and data integration pipelines for large structured and unstructured datasets.
  • Migrate and modernize Hadoop-based data, analytics, and AI/ML workloads to AWS, Azure, and Databricks.
  • Automate infrastructure provisioning, platform deployment, and operational processes using Terraform and CloudFormation.
  • Implement data governance, metadata management, lineage, security, and access management capabilities across cloud platforms.
  • Monitor and optimize platform performance, scalability, reliability, operational efficiency, and cost across cloud environments.
  • Partner with architecture, product, security, application, and engineering teams to onboard workloads and translate enterprise designs into production implementations.
  • Troubleshoot complex production issues, improve platform reliability, and promote operational excellence.
  • Provide technical guidance and mentorship while recommending improvements in cloud, data, and platform engineering technologies.

Requirements

  • At least 8 years of experience in cloud data engineering, big data engineering, platform engineering, or data platform engineering.
  • Strong hands-on experience building and operating enterprise-scale cloud data platforms.
  • Expertise with AWS data platform services including EMR, EMR on EKS, Glue, Athena, S3, IAM, and Lake Formation, and Azure services including ADLS, Azure Databricks, Synapse Analytics, Data Factory, Azure Kubernetes Service, and identity and access management.
  • Hands-on experience with Databricks, Delta Lake, Unity Catalog, and Lakehouse architecture patterns.
  • Experience designing and operating data platform solutions on at least one major cloud platform, with preference for AWS, Azure, or both.
  • Experience migrating workloads from Hadoop-based platforms to cloud environments.
  • Strong proficiency in Python, Scala, or Java.
  • Hands-on experience with Infrastructure-as-Code tools such as Terraform or CloudFormation.
  • Strong expertise in Spark, Hive, Hadoop ecosystem technologies, and distributed data processing frameworks.
  • Experience implementing automation, CI/CD, platform engineering practices, data governance, security, compliance, metadata management, and access controls.
  • Strong Linux, operating system, networking, and troubleshooting skills.
  • Experience working with architecture teams to translate enterprise architecture designs into scalable, production-ready implementations.
  • Preferred qualifications include experience with multi-tenant enterprise data platforms, Kubernetes, EKS, containerized workloads, observability, monitoring, logging, alerting, reliability engineering, cost governance, cloud optimization, AI/ML platforms, data science workloads, and large-scale cloud transformation programs.
  • Experience mentoring engineers and driving engineering best practices across teams is preferred.

Tech Stack

Apache HadoopApache HiveApache SparkAWSAzureDatabricksJavaKubernetesLinuxPythonScalaTerraform

Categories

Data EngineeringDevOps
Visa

About Visa

10,000+ employees

Visa builds and operates a global digital payments network that connects consumers, merchants, financial institutions, and governments. It earns fees for processing card transactions and offering risk, data, and value‑added services through VisaNet and products like tokenization and tap‑to‑pay. Founded in 1958 and headquartered in Foster City, California, Visa is a public company (NYSE: V) serving clients in more than 200 countries and territories.

Contact me