Lead Cloud Engineer
Mastercard21 hours ago
Lisbon, PortugalStaff+
Responsibilities
- Design, develop, and support large-scale cloud-native data solutions on AWS and Databricks.
- Build and maintain batch, streaming, and real-time data pipelines for analytics, reporting, machine learning, and AI workloads.
- Engineer scalable distributed data processing systems using Apache Spark and related Big Data technologies.
- Design and implement Lakehouse architectures using Databricks, Delta Lake, and Apache Iceberg.
- Lead migrations from Hadoop-based platforms to modern cloud-native architectures.
- Develop reusable frameworks and engineering patterns that improve scalability, reliability, and operational efficiency.
- Partner with architects, product managers, data scientists, and software engineers to define and implement cloud data solutions.
- Evaluate approaches for data ingestion, transformation, storage, governance, and consumption.
- Implement security, governance, performance optimization, observability, and cost-management practices.
- Automate infrastructure and delivery using Infrastructure-as-Code and CI/CD pipelines.
- Contribute to Mastercard Marketing Services’ cloud and data strategy and future AI and machine learning capabilities.
Requirements
- Strong hands-on experience with Hadoop, HDFS, Hive, Spark, and distributed computing frameworks.
- Deep AWS cloud data engineering experience, including storage, compute, networking, security, and scalable data architectures.
- Experience designing modern data solutions with Databricks, Spark, Delta Lake, and cloud-native technologies.
- Proficiency in Python, Java, or Scala for large-scale distributed data processing applications.
- Experience building robust batch and streaming pipelines for high-volume and high-velocity workloads.
- Understanding of Lakehouse architecture and open-table formats such as Apache Iceberg and Delta Lake.
- Skills in data modeling, schema design, partitioning, query optimization, and performance tuning.
- Familiarity with enterprise data governance, security, privacy, compliance, and structured and unstructured data.
- Experience with Infrastructure-as-Code such as Terraform, CI/CD, and cloud-native monitoring and observability.
- Strong analytical, problem-solving, communication, and collaboration skills across technical and business teams.
- Bachelor’s or master’s degree in Computer Science, Software Engineering, Information Systems, or a related field.
- Preferred: experience with Databricks on AWS in enterprise production environments and migrations from Hadoop or Cloudera ecosystems.
- Preferred: hands-on experience with Apache Kafka, Flink, or other event-streaming technologies.
- Preferred: experience supporting machine learning, MLOps, or AI-driven data platforms.
- Preferred: AWS or Databricks certifications, FinOps and cloud cost optimization familiarity, and petabyte-scale data experience.
Tech Stack
Categories
Data Engineering
About Mastercard
Mastercard builds and operates a global payments network used by banks, merchants, fintechs, and governments, offering card processing, real-time payments, tokenization, and fraud/risk services. It generates revenue from transaction processing and assessment/service fees across more than 200 countries and territories. Founded in 1966 and headquartered in Purchase, New York, Mastercard is a public company listed on the NYSE (ticker: MA).