Senior Backend Engineer
Aivar Innovations Private Limited2 months ago
Coimbatore, IndiaMid Level / Senior
Responsibilities
- Design a pipeline orchestration framework supporting asynchronous parallel and sequential stage execution, retries, and dead-letter queues.
- Build connectors for more than 50 enterprise APIs, databases, file systems, and SaaS platforms.
- Implement PII/PHI detection, data integrity checks, format compliance, and input sanitization.
- Develop reconstruction workflows that transform extracted data into Markdown, vector representations, and context maps.
- Design event-driven architecture with message queuing, backpressure handling, and graceful degradation.
- Implement immutable audit-trail logging across all six processing stages.
- Build multi-tenant data isolation and strict customer separation throughout the pipeline.
- Support production testing, CI/CD, monitoring, and incident response.
Requirements
- 4–8 years of software engineering experience, including at least 2 years building data-intensive pipelines.
- Experience building distributed Python systems with asyncio, aiohttp, and Celery rather than scripting.
- Experience with Kafka, SQS, RabbitMQ, or Redis Streams for messaging and streaming.
- Experience with high-volume pipelines processing 100K or more documents or records per batch.
- Knowledge of REST and GraphQL API design, versioning, rate limiting, and authentication.
- Experience with AWS services including Lambda, SQS, SNS, MSK, EKS, Step Functions, ECS, S3, DynamoDB, and RDS.
- Experience with multi-tenant SaaS architecture and customer data isolation patterns.
- Experience with comprehensive testing, CI/CD, monitoring, and incident response.
Tech Stack
Amazon DynamoDBApache KafkaDockerElasticsearchFastAPIGitHub ActionsGrafanaPostgreSQLPrometheusPythonRabbitMQRedisTerraform
Categories
BackendData Engineering