17 days ago
Pune, IndiaMid Level
Responsibilities
- Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads.
- Manage and monitor Spark Streaming jobs, including eligibility and data quality checks, Kafka publishing, and exception request processing.
- Manage and monitor scheduled Spark Batch jobs that store audit, inbound, and outbound data in Hive tables.
- Respond to ITRS alerts and system monitoring events such as job failures, terminated jobs, and missed schedules.
- Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server.
- Configure and manage JDBC connections from SAS, Python, and R to sandbox and data platform environments.
- Collaborate on schema creation, permission grants, and country-specific schema access during code deployment.
- Support retirement of legacy applications and adoption of modern data engineering technologies.
Requirements
- 2–5 years of relevant experience.
- Proficiency in Python and PySpark for distributed data engineering.
- Production experience with Apache Spark, including Streaming and Batch.
- Strong SQL knowledge and experience with Oracle, Impala, and Hive databases.
- Familiarity with Kafka or event-driven architectures.
- Experience with job scheduling and monitoring tools such as Autosys and ITRS.
- Knowledge of data governance, access management, and information security standards.
- Experience diagnosing and resolving complex data pipeline issues.
- Experience with SAS or equivalent enterprise analytical platforms is preferred.
- Familiarity with SDLC practices across Development, UAT, and Production environments is preferred.
- Bachelor’s degree or university degree in information technology or engineering, or equivalent experience.
- Working knowledge of programming languages, business application programming and debugging, industry practices, and standards.
Benefits
- Full-time position within Citi’s Technology organization.
Tech Stack
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.