23 hours ago
Pune, India or Chennai, IndiaSenior
Responsibilities
- Architect, develop, and maintain reusable enterprise data services for ingestion, transformation, standardization, validation, reconciliation, and distributed processing.
- Build scalable batch, near-real-time, and streaming integration frameworks connecting files, databases, APIs, Kafka, CDC, and enterprise messaging systems.
- Develop reconciliation and matching engines supporting complex rules, multi-pass matching, tolerance handling, balance proofing, exception generation, and one-to-many or many-to-many matching.
- Engineer data control and integrity frameworks for completeness, accuracy, timeliness, control totals, source-to-target validation, auditability, lineage, replay, and recovery.
- Develop configurable data quality services for schema validation, profiling, anomaly detection, scoring, exception handling, and automated remediation.
- Build and optimize Databricks and lakehouse workloads using distributed processing, event-driven services, stateful streaming, checkpointing, deduplication, and reliable event delivery.
- Design REST APIs, microservices, metadata-driven processing frameworks, configuration services, and reusable libraries.
- Implement AWS infrastructure, Terraform, CI/CD, automated testing, observability, performance optimization, FinOps, production support, and disaster recovery.
- Integrate AI/ML capabilities for intelligent matching, anomaly detection, data quality remediation, metadata discovery, and AI-assisted data processing.
- Own production system performance, reliability, scalability, resilience, and automated recovery.
Requirements
- Bachelor’s or university degree, or equivalent experience; a master’s degree is preferred.
- At least 8 years of hands-on software development and data platform engineering experience.
- Strong hands-on development experience with Scala, Java, or Python, including object-oriented design, concurrency, distributed systems, and software design patterns.
- Strong experience with Apache Spark, Scala Spark, PySpark, Spark SQL, Structured Streaming, and performance tuning.
- Experience with AWS, Databricks, S3, Apache Iceberg, Delta Lake, and Unity Catalog.
- Experience with Kafka/MSK, CDC, REST APIs, JDBC, file processing, and enterprise messaging.
- Experience developing data transformation, reconciliation, rule-engine, validation, data-control, data-quality, and exception-processing services.
- Experience with microservices, REST APIs, Spring Boot or FastAPI, metadata-driven frameworks, and service integration.
- Experience with SQL, data modeling, schema evolution, partitioning, incremental processing, and data optimization.
- Experience with Terraform, Docker, Git, CI/CD, automated testing, CloudWatch, OpenTelemetry, and production troubleshooting.
- Proven experience building complex, high-throughput distributed data services and reusable frameworks rather than only implementing ETL pipelines.
- Demonstrated ownership of production systems, performance, reliability, and scalability.
- Experience with enterprise reconciliation or financial data processing platforms, real-time matching, balance proofing, and stateful stream processing is desirable.
- Experience with AI/ML, MLflow, vector search, RAG, agentic AI integration, MongoDB Atlas, PostgreSQL/Aurora, enterprise workflow integration, financial services data governance, regulatory controls, or operational resilience is desirable.
- Experience delivering enterprise-scale platforms in regulated environments is preferred.
Tech Stack
Apache KafkaApache SparkAWSDatabricksDockerFastAPIGitJavaMLflowPostgreSQLPythonScalaSpring BootSQLTerraform
Categories
BackendData Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.
