3 months ago
Pune, IndiaSenior / Staff+
Responsibilities
- Conduct feasibility studies, time and cost estimates, IT planning, risk assessment, application development, and model development.
- Design, build, test, implement, monitor, and control application systems and programs.
- Develop and optimize Spark and PySpark applications for distributed data processing.
- Use HDFS, YARN, MapReduce, Spark, SQL, and distributed databases to support data applications.
- Provide user and operational support, consult with business users and technology groups, and recommend programming solutions.
- Analyze complex application and business-process issues and recommend system and security improvements.
- Define operating standards and processes and ensure required procedures are followed.
- Serve as an advisor or coach to new or lower-level analysts and act as a subject-matter expert for stakeholders.
- Assess and escalate risks while maintaining compliance with applicable laws, rules, regulations, and company policies.
Requirements
- 5–8 years of relevant experience.
- Experience in systems analysis and software application programming.
- Experience managing and implementing successful projects.
- Working knowledge of consulting and project management techniques and methods.
- Bachelor’s or university degree, or equivalent experience.
- In-depth understanding of HDFS architecture, data storage, and fault-tolerance mechanisms, including HDFS commands and administration.
- Strong understanding of YARN resource management, job scheduling, and the MapReduce programming paradigm.
- Knowledge of ZooKeeper for distributed coordination services.
- Strong proficiency in Spark Core, Spark SQL, and Spark Streaming; Spark GraphX knowledge is beneficial.
- Expert-level Python and PySpark programming skills for developing Spark applications.
- Experience with Spark performance optimization, including caching, partitioning, shuffle optimization, and memory management.
- Familiarity with Pandas, NumPy, scripting for automation, and data orchestration.
- Ability to write and tune complex SQL queries using subqueries and window functions.
- Knowledge of HBase, Cassandra, MongoDB, or similar distributed data stores.
- Familiarity with RDBMS concepts and SQL-based data integration.
- Understanding of dimensional modeling, fact and dimension tables, and star and snowflake schemas.
- Ability to work under pressure and manage deadlines or unexpected changes in requirements.
Benefits
- Full-time position.
Tech Stack
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.