28 days ago
Pune, IndiaSenior / Staff+
Responsibilities
- Conduct feasibility studies, time and cost estimates, IT planning, risk assessment, application development, and model development.
- Design, build, test, implement, monitor, and control application systems and programs.
- Develop and optimize Spark and PySpark applications for distributed data processing.
- Use HDFS, YARN, MapReduce, Spark, SQL, and distributed databases to support data applications.
- Provide user and operational support, consult with business users and technology groups, and recommend programming solutions.
- Analyze complex application and business-process issues and recommend system and security improvements.
- Define operating standards and processes and ensure required procedures are followed.
- Serve as an advisor or coach to new or lower-level analysts and act as a subject-matter expert for stakeholders.
- Assess and escalate risks while maintaining compliance with applicable laws, rules, regulations, and company policies.
Requirements
- 5–8 years of relevant experience.
- Experience in systems analysis and software application programming.
- Experience managing and implementing successful projects.
- Working knowledge of consulting and project management techniques and methods.
- Bachelor’s or university degree, or equivalent experience.
- In-depth understanding of HDFS architecture, data storage, and fault-tolerance mechanisms, including HDFS commands and administration.
- Strong understanding of YARN resource management, job scheduling, and the MapReduce programming paradigm.
- Knowledge of ZooKeeper for distributed coordination services.
- Strong proficiency in Spark Core, Spark SQL, and Spark Streaming; Spark GraphX knowledge is beneficial.
- Expert-level Python and PySpark programming skills for developing Spark applications.
- Experience with Spark performance optimization, including caching, partitioning, shuffle optimization, and memory management.
- Familiarity with Pandas, NumPy, scripting for automation, and data orchestration.
- Ability to write and tune complex SQL queries using subqueries and window functions.
- Knowledge of HBase, Cassandra, MongoDB, or similar distributed data stores.
- Familiarity with RDBMS concepts and SQL-based data integration.
- Understanding of dimensional modeling, fact and dimension tables, and star and snowflake schemas.
- Ability to work under pressure and manage deadlines or unexpected changes in requirements.
Benefits
- Full-time position.
Tech Stack
Categories
Data Engineering
About Citi
Citi's mission is to serve as a trusted partner to our clients by responsibly providing financial services that enable growth and economic progress. Our core activities are safeguarding assets, lending money, making payments and accessing the capital markets on behalf of our clients. We have over 200 years of experience helping our clients meet the world's toughest challenges and embrace its greatest opportunities. We are Citi, the global bank – an institution connecting millions of people across hundreds of countries and cities. For information on Citi’s commitment to privacy, visit on.citi/privacy.