Lead Data Engineer
Mastercard2 months ago
Base Salary
$140k - $231k/yr
Responsibilities
- Lead the design, development, and evolution of enterprise-grade data platforms and pipelines for credit risk products, decisioning, analytics, and regulatory reporting.
- Architect and implement scalable ETL/ELT frameworks using Databricks, Apache Spark, Delta Lake, and cloud-native technologies.
- Drive cloud transformation and migration of legacy data assets to modern cloud-based Data Lakehouse architectures.
- Establish data quality, lineage, metadata, governance, observability, and monitoring capabilities.
- Translate complex business requirements into scalable technical solutions with cross-functional teams.
- Define engineering standards for coding, testing, data quality, deployment automation, documentation, and operational excellence.
- Lead technical design reviews and influence architecture for data-intensive applications, platforms, and services.
- Optimize data workloads and platforms for performance, reliability, scalability, resiliency, and cost efficiency.
- Build reusable, governed data products supporting AI, machine learning, advanced analytics, and risk use cases.
- Drive technical initiatives from strategy through execution while managing dependencies, risks, and alignment with platform strategy.
- Mentor and coach engineers and provide technical guidance, design expertise, knowledge sharing, and feedback.
- Shape technology roadmaps and delivery priorities across the Enterprise Credit Risk portfolio.
- Ensure solutions meet security, privacy, regulatory, compliance, and audit requirements.
- Communicate technical trade-offs, risks, dependencies, and recommendations to technical and business stakeholders.
Requirements
- Deep expertise designing and implementing large-scale data engineering solutions and distributed data processing systems.
- Advanced proficiency with Databricks, Apache Spark/PySpark, Delta Lake, SQL, and Python.
- Experience building and operating cloud-based data platforms using Azure, AWS, or GCP.
- Strong experience developing enterprise-grade ETL/ELT pipelines, data integration frameworks, and streaming architectures.
- Strong understanding of data modeling and data architecture for analytical and operational workloads.
- Experience implementing data quality, lineage, metadata management, observability, and data governance frameworks.
- Experience with Parquet, Avro, ORC, workflow orchestration, automated testing, Git/GitHub, and DevOps practices.
- Strong understanding of security, privacy, regulatory, and compliance requirements for sensitive financial and customer data.
- Ability to lead complex technical initiatives across multiple teams and influence engineering direction without direct authority.
- Demonstrated ability to mentor engineers and elevate technical capabilities.
- Excellent communication and stakeholder management skills.
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related STEM discipline, or equivalent practical experience.
- Preferred experience includes Hadoop, Azure Data Services, Kafka/event streaming, GitHub and CI/CD tooling, and cloud-based Data Lakehouse architectures.
- Java-based application development experience is a strong plus.
Benefits
- Hybrid role based in O’Fallon, Missouri, requiring three days per week onsite.
- Full-time benefits generally include medical, prescription drug, dental, vision, disability, and life insurance; flexible spending and health savings accounts; paid leave; vacation, personal, sick, and holiday time; 401(k) matching; fitness benefits; tuition reimbursement; and other benefits.
Tech Stack
Apache AirflowApache HadoopApache KafkaApache SparkAWSAzureDatabricksGitGoogle Cloud PlatformJavaPythonSQL
Categories
Data Engineering
About Mastercard
Mastercard builds and operates a global payments network used by banks, merchants, fintechs, and governments, offering card processing, real-time payments, tokenization, and fraud/risk services. It generates revenue from transaction processing and assessment/service fees across more than 200 countries and territories. Founded in 1966 and headquartered in Purchase, New York, Mastercard is a public company listed on the NYSE (ticker: MA).