1 day ago
Pune, IndiaStaff+
Responsibilities
- Lead applications systems analysis and programming activities in coordination with technology and management teams.
- Design and implement new or revised application systems, products, process improvements, and system enhancements.
- Resolve high-impact problems through detailed analysis of complex business and system processes and industry standards.
- Ensure application designs follow the overall architecture blueprint and establish coding, testing, debugging, and implementation standards.
- Analyze business, architecture, and infrastructure integration to support business goals and develop innovative solutions.
- Serve as an advisor or coach to mid-level developers and analysts and allocate work as needed.
- Integrate model test suites into CI/CD pipelines and conduct AI/LLM evaluation, adversarial testing, prompt-injection defense testing, safety-guardrail testing, and PII-masking validation.
- Assess and manage business, compliance, operational, and technology risks while safeguarding the firm, clients, and assets.
Requirements
- 10+ years of relevant experience in application development or systems analysis.
- Extensive experience with systems analysis and software application programming.
- Experience managing and implementing successful projects and serving as a subject matter expert in at least one area of application development.
- Strong proficiency in Python and standard data and testing ecosystems such as pytest, NumPy, pandas, and scikit-learn.
- Deep practical experience with AI/LLM evaluation tools and libraries such as Ragas, DeepEval, Promptfoo, TruLens, Giskard, LangSmith, or Weights & Biases.
- Familiarity with foundation model APIs including OpenAI, Anthropic Claude, Azure OpenAI, Google Vertex AI, and Hugging Face, as well as LangChain and LlamaIndex.
- Hands-on experience with CI/CD and MLOps platforms including GitHub Actions, GitLab CI, Tekton, Jenkins, MLflow, and Kubeflow.
- Knowledge of OWASP Top 10 for LLMs, prompt-injection defense testing, safety guardrails such as NeMo Guardrails and Llama Guard, and PII-masking validation.
- Demonstrated leadership, project management, prioritization, and clear written and verbal communication skills.
- Bachelor’s or university degree or equivalent experience required; a master’s degree is preferred.
Tech Stack
Categories
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.