2 days ago
Responsibilities
- Design and implement large-scale data warehouses, data lakes, and lakehouses on AWS.
- Build modular and resilient data pipelines using native AWS services.
- Implement data governance and cataloging, ETL and metadata frameworks, serverless processing, distributed workloads, real-time pipelines, and pipeline orchestration.
- Design and optimize Amazon Redshift schemas and query performance, including Spectrum and Serverless features.
- Manage operational databases and integrate or migrate data across AWS and other data systems.
- Apply dimensional, normalized, and lakehouse data modeling patterns.
- Support legacy and on-premises to AWS migration engagements.
- Collaborate with globally distributed teams in delivery environments.
Requirements
- 8+ years of experience designing and implementing AWS data warehouses and data lakes or lakehouses.
- Deep working knowledge of AWS data and analytics services including Amazon S3, AWS Lake Formation, AWS Glue, AWS Lambda, Amazon EMR, AWS Kinesis, AWS Step Functions, Amazon MWAA, Amazon Redshift, Amazon Athena, Amazon RDS, AWS DMS, AWS Glue Connectors, Amazon EventBridge, Amazon SNS, and Amazon SQS.
- Strong programming capability in Python and PySpark for large-scale data processing.
- Proficiency writing complex SQL queries, analytical functions, and performance-tuning queries for large datasets.
- Experience with partitioning, indexing, scaling, query optimization, data modeling, and resilient pipeline architecture.
- Familiarity with Amazon DynamoDB, MongoDB, or Amazon DocumentDB.
- Solution Architect - Associate or Data Engineer - Associate certification.
- Proven success working with globally distributed teams in collaborative delivery environments.
- Preferred experience with enterprise greenfield projects involving application development workloads, FSI customer workloads, Denodo, Collibra, and SSIS.
Benefits
- Full-time, fully remote position based in the UK.
- Work with Fortune 500 companies and disruptive innovators on large-scale AI, cloud, and data transformation projects.
- Gain hands-on experience with AI, ML, data, and cloud technologies in a research-driven environment.
- Continuous upskilling and collaboration with a global team at a certified Great Place to Work company.
Tech Stack
Categories
Data Engineering
About Quantiphi
Quantiphi is an award-winning AI-first digital engineering company driven by the desire to reimagine and realize transformational opportunities at the heart of the business. Since its inception in 2013, Quantiphi has solved the toughest and most complex business problems by combining deep industry experience, disciplined cloud, and data-engineering practices, and cutting-edge artificial intelligence research to achieve accelerated and quantifiable business results. Learn more at www.quantiphi.com.