1 day ago
Chennai, IndiaMid Level / Senior
Responsibilities
- Design, implement, and maintain scalable data ingestion and ETL/ELT pipelines using Apache Spark and Java.
- Develop low-latency microservices and distributed processing modules integrated with messaging platforms.
- Build interfaces to relational databases, data lakes, and NoSQL stores.
- Profile, debug, and optimize Spark jobs, including partitioning, caching, memory allocation, serialization, query execution, and data skew.
- Monitor distributed workloads on Kubernetes, Apache YARN, and cloud-native orchestration platforms.
- Design data storage schemas and implement validation, cleansing, governance, error handling, encryption, and access controls.
- Write unit, integration, and automated regression tests.
- Configure and maintain CI/CD pipelines and participate in code reviews and software development ceremonies.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Software Engineering, or a related technical discipline.
- 3–6 years of professional software engineering experience.
- 2–4 years of experience building scalable distributed data processing applications with Java and Apache Spark.
- Deep proficiency in Core Java, including multithreading, concurrency, object-oriented programming, memory management, and JVM internals.
- Hands-on experience with Apache Spark RDDs, DataFrames, Datasets, Spark SQL, and Structured Streaming.
- Strong knowledge of distributed architectures, HDFS, YARN, Hive, messaging platforms, relational databases, and NoSQL datastores.
- Advanced SQL skills and proficiency with Maven, Gradle, and Git.
- Experience with testing using JUnit, Mockito, Spark Testing Base, and distributed testing patterns.
- Preferred experience with AWS, Azure, Google Cloud, Apache Iceberg, Delta Lake, Apache Hudi, Docker, Kubernetes, Apache Airflow, Luigi, Scala, or Python/PySpark.
Tech Stack
Apache AirflowApache CassandraApache HBaseApache HiveApache KafkaApache SparkAWSAzureDatabricksDockerGitGitHub ActionsGitLab CI/CDGoogle BigQueryGoogle CloudGradleJavaJenkinsJUnitKubernetesMavenMongoDBMySQLPostgreSQLPythonRabbitMQScalaSnowflakeSQL
Categories
BackendData Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.