1 month ago
Pune, IndiaStaff+
Responsibilities
- Design, build, and evolve the data architecture supporting greenfield Generative AI products across Markets.
- Develop scalable Python pipelines to ingest, validate, transform, enrich, and integrate billions of records from internal and external sources.
- Build curated, queryable datasets and high-performance serving layers for low-latency application access.
- Design and optimize storage, indexing, partitioning, ingestion, transformation, and query performance, initially using PostgreSQL and Parquet.
- Implement controls for data quality, reconciliation, lineage, schema evolution, idempotency, failure recovery, data classification, entitlements, access controls, and security.
- Build production observability into pipelines through metrics, logging, alerting, and operational diagnostics.
- Contribute directly to code, architecture reviews, technical standards, automated tests, CI/CD pipelines, and the team’s engineering direction.
- Collaborate with AI engineers, software engineers, architects, product partners, and Markets stakeholders.
- Use Devin and GitHub Copilot to improve development quality and productivity.
Requirements
- Extensive hands-on experience in data engineering, software engineering, or a closely related discipline.
- Deep practical expertise in Python and the ability to build maintainable, production-grade software.
- Experience designing and delivering large-scale data platforms or data-intensive applications.
- Strong SQL skills and extensive experience with relational databases, particularly PostgreSQL.
- Experience with data modeling, indexing, partitioning, query optimization, and database performance tuning.
- Knowledge of the Python data ecosystem, including pandas, PyArrow, SQLAlchemy, and NumPy.
- Experience with columnar formats such as Apache Parquet and efficient storage and serialization strategies.
- Experience processing large-scale datasets using Apache Spark, Dask, Polars, or equivalent frameworks.
- Strong understanding of ETL and ELT architecture, including incremental processing, idempotency, failure recovery, and schema evolution.
- Experience implementing automated data quality controls, reconciliation, lineage, monitoring, and operational alerting.
- Strong software engineering fundamentals in design, testing, maintainability, code review, and CI/CD.
- Experience deploying and operating services on container platforms such as Kubernetes or OpenShift.
- Understanding of data governance and security practices, including encryption, masking, classification, entitlements, and fine-grained access control.
- Ability to operate in ambiguity, take ownership, influence technical direction, communicate effectively, and collaborate with engineering partners.
- Preferred experience with Apache Airflow, Dagster, or Prefect; Generative AI applications, retrieval architectures, LLMs, agentic systems, or structured evaluation; and data foundations for search, retrieval, analytics, machine learning, or AI applications.
- Preferred knowledge of financial instruments, trading concepts, and data structures in FX, Equities, or other capital markets domains, as well as temporal, reference, market, or transactional data.
- A bachelor’s or master’s degree in Computer Science, Engineering, or another relevant quantitative discipline, or equivalent professional experience.
Tech Stack
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.