1 hour ago
Bengaluru, IndiaMid Level
Responsibilities
- Develop and maintain software applications and trading systems that process real-time market data and support trading strategies.
- Build AWS-based ETL/ELT pipelines using Glue, Step Functions, Lambda, and DMS, including direct-to-Aurora snapshot and delta ingestion.
- Implement referential integrity, correct ordering, idempotency, recovery, monitoring, validation, and reconciliation frameworks.
- Manage scaling and throughput during high-volume end-of-day ingestion peaks.
- Automate deployments, workflows, and moderately complex tests, and perform unit, integration, performance, and debugging activities.
- Monitor live trading systems, resolve operational issues, support production readiness and disaster recovery, and maintain runbooks and technical documentation.
- Collaborate with traders, analysts, quality assurance teams, and cross-functional stakeholders to deliver and improve trading solutions.
Requirements
- At least 3 years of relevant work experience, typically reflecting 5 or more years in data engineering, data integration, or related roles.
- Proven experience building resilient, idempotent ingestion frameworks and supporting high-volume, time-sensitive processing, especially end-of-day workloads.
- Expert-level SQL development and performance optimization skills, plus strong proficiency in Python.
- Hands-on experience with ETL pipelines and orchestration frameworks and a solid understanding of ACID-compliant data ingestion principles.
- Experience with schema evolution, Change Data Capture patterns, production readiness, and end-to-end solution delivery.
- Strong operational mindset involving monitoring, alerting, runbooks, and production support.
- Preferred experience with SAP ODP, enterprise data replication technologies, distributed PostgreSQL systems, event-driven architectures such as Kafka, and cloud-native AWS platforms.
Tech Stack
Categories
Data Engineering
