2 months ago
Bengaluru, IndiaMid Level
Responsibilities
- Develop and maintain software applications and trading systems that process real-time market data and support trading strategies.
- Build AWS-based ETL/ELT pipelines using Glue, Step Functions, Lambda, and DMS, including direct-to-Aurora snapshot and delta ingestion.
- Implement referential integrity, correct ordering, idempotency, recovery, monitoring, validation, and reconciliation frameworks.
- Manage scaling and throughput during high-volume end-of-day ingestion peaks.
- Automate deployments, workflows, and moderately complex tests, and perform unit, integration, performance, and debugging activities.
- Monitor live trading systems, resolve operational issues, support production readiness and disaster recovery, and maintain runbooks and technical documentation.
- Collaborate with traders, analysts, quality assurance teams, and cross-functional stakeholders to deliver and improve trading solutions.
Requirements
- At least 3 years of relevant work experience, typically reflecting 5 or more years in data engineering, data integration, or related roles.
- Proven experience building resilient, idempotent ingestion frameworks and supporting high-volume, time-sensitive processing, especially end-of-day workloads.
- Expert-level SQL development and performance optimization skills, plus strong proficiency in Python.
- Hands-on experience with ETL pipelines and orchestration frameworks and a solid understanding of ACID-compliant data ingestion principles.
- Experience with schema evolution, Change Data Capture patterns, production readiness, and end-to-end solution delivery.
- Strong operational mindset involving monitoring, alerting, runbooks, and production support.
- Preferred experience with SAP ODP, enterprise data replication technologies, distributed PostgreSQL systems, event-driven architectures such as Kafka, and cloud-native AWS platforms.
Tech Stack
Categories
Data Engineering
About Cargill
Cargill builds and operates global food and agriculture supply chains, selling commodities, food ingredients, animal nutrition, and risk-management services to food manufacturers, retailers, and farmers. It trades and processes grains, oilseeds, cocoa, and proteins, and provides logistics and biofuel solutions. Founded in 1865 and headquartered in Wayzata, Minnesota, Cargill is privately held and family-owned.