Responsibilities
- Gather data requirements and deliver technical solutions with data engineers, architects, product teams, and business stakeholders.
- Participate in requirements gathering, solution design, development, testing, deployment, and operational support for data engineering initiatives.
- Design, develop, and maintain near-real-time data pipelines and Flink-based data lake solutions using Kafka, Flink, Python, and SQL.
- Build and support data ingestion, transformation, schema evolution, partitioning, compaction, and data lifecycle management in Iceberg-based data lakes.
- Develop analytical datasets, data models, and DataMart solutions for reporting and business intelligence.
- Build and optimize distributed data processing and query workloads using Spark, Trino, StarRocks, and SQL.
- Implement data validation, quality checks, monitoring, alerting, and observability across pipelines and data lake workloads.
- Troubleshoot production data issues, perform initial root-cause analysis, and support incident resolution.
- Write maintainable, testable, and documented code; participate in code reviews and follow engineering standards.
- Contribute to technical documentation, operational runbooks, and continuous improvement in a collaborative Agile environment.
Requirements
- 2–4 years of professional experience in Data Engineering, Software Engineering, Analytics Engineering, or a related discipline.
- Hands-on programming experience with Python and SQL, plus understanding of data structures, algorithms, relational databases, and software development principles.
- Understanding of distributed systems, data processing, and large-scale data architectures.
- Experience building, maintaining, or supporting ETL and ELT data pipelines and working with data warehouses, data lakes, or lakehouse platforms.
- Understanding of data modeling, dimensional modeling, DataMart development, SQL query optimization, data partitioning, indexing, and performance tuning.
- Experience with batch processing and exposure to streaming or event-driven data pipelines.
- Experience with relational databases and familiarity with NoSQL databases.
- Hands-on experience with AWS and one or more of Amazon S3, Amazon EMR, Amazon Redshift, or Amazon MSK.
- Practical experience with technologies such as Apache Spark/PySpark, Apache Airflow, Apache Kafka, Debezium, Apache Flink, Apache Iceberg, Trino, StarRocks, ClickHouse, or Hadoop.
- Experience with Git, code reviews, testing, and CI/CD practices.
- Familiarity with Kafka Connect, Maxwell, binlog/WAL/GTID log replication, change data capture, database replication, or incremental data processing.
- Ability to troubleshoot issues, analyze problems, communicate solutions clearly, and collaborate effectively.
- Preferred qualifications include familiarity with data quality frameworks, pipeline monitoring, observability, Docker, Kubernetes, infrastructure automation, DevOps practices, on-premises or hybrid data platforms, lakehouse architecture, open table formats, fintech or high-volume transaction environments, Cassandra, MongoDB, or Redis.
Benefits
- Collaborative and inclusive workplace with opportunities to work on modern data technologies at global scale.
- Access to more than 5,000 training courses through learning partners including Harvard, Coursera, and Udacity.
- Opportunities to work with experienced engineers and grow across data engineering, cloud, and distributed systems.
- Opportunity to learn and innovate in a global fintech environment.
Tech Stack
Categories
About PayU
Global reach, local expertise: we combine the best of both worlds to reshape the future of payments and fintech to build a world of unlimited possibilities. We successfully bridge the global and local gap by supplying ultimate global payment coverage with extensive local expertise and support. Our vision is to create a world without financial borders where everyone can prosper. Thus, we strive to shape the payment industry of tomorrow to support global financial inclusion. As the payments and fintech business of Prosus- a global consumer internet group and one of the largest technology investors in the world- we are a leading online payment service provider operating in 50+ high-growth markets, dedicated to creating cutting-edge financial services tailored to the needs of over 450,000 merchants and millions of consumers. Focused on empowering people through financial services and creating a world without financial borders, we have set out to be one of the most prominent investors in the Fintech space, with investments totaling over $1 billion to date. We deploy more than 400 payment methods and PCI-certified platforms to process approximately 10 million payments daily. As a Fintech innovator, we develop cutting-edge technologies to ensure that merchants benefit from the highest approval rates and can optimally expand their global reach.