2 days ago
Pune, IndiaSenior
Responsibilities
- Design and maintain batch and incremental data ingestion pipelines from APIs, relational databases, SaaS applications, and file sources.
- Build AWS ingestion solutions using Amazon S3, AWS Glue, Lambda, IAM, and Step Functions.
- Develop Snowflake data models and transformation pipelines using SQL, Python, dbt, or equivalent tools.
- Implement Bronze, Silver, and Gold layers, data contracts, lineage, change data capture, incremental loading, deduplication, schema evolution, and historical data management.
- Build data quality checks, reconciliation controls, monitoring, alerting, recovery procedures, and automated testing for data pipelines.
- Optimize Snowflake queries, warehouse usage, and storage for performance and cost control.
- Implement access controls, encryption, secrets management, and data protection requirements.
- Establish repeatable deployment practices, troubleshoot production issues, perform root cause analysis, and improve pipeline reliability.
- Collaborate with analysts, data scientists, architects, and business stakeholders to translate requirements into technical solutions.
- Maintain technical documentation and provide engineering guidance through design and code reviews.
Requirements
- 8+ years of professional experience in data engineering, data warehousing, ETL/ELT development, or related roles.
- Strong hands-on production experience with AWS and Snowflake.
- Advanced SQL skills, including complex transformations, analytical functions, and query optimization.
- Strong Python skills for API integrations, data processing, automation, and error handling.
- Experience building ingestion pipelines with pagination, authentication, rate limiting, retries, and checkpointing.
- Experience with Snowflake stages, storage integrations, COPY INTO, Snowpipe, Streams, and Tasks.
- Experience with dbt or equivalent SQL-based transformation tooling and Apache Airflow, AWS Step Functions, or equivalent orchestration platforms.
- Strong understanding of dimensional modeling, slowly changing dimensions, and incremental processing.
- Experience implementing data quality, observability, lineage, and operational support practices.
- Proficiency with Git, code reviews, and CI/CD.
- Strong problem-solving, communication, and stakeholder collaboration skills.
- Preferred experience preparing AI/ML datasets, feature engineering, training and inference pipelines, and MLOps workflows.
- Preferred familiarity with experiment tracking, model versioning, deployment, monitoring, retraining, model drift, data drift, and reproducibility.
- Preferred experience with Amazon SageMaker, MLflow, Snowpark, Amazon Kinesis, Apache Kafka, Terraform, or AWS CloudFormation.
- AWS, Snowflake, or relevant data engineering certifications are preferred.
Benefits
- Competitive salary and opportunity to earn an annual bonus.
- Career growth and development opportunities in a global organization.
- Flexible approach to work; the role is based in Pune, India and is work from home.
Categories
Data Engineering
About Convera
Convera provides B2B cross-border payments, foreign exchange, and risk/compliance services through software and integrations for businesses of all sizes. It serves more than 30,000 customers across 200 countries and 140+ currencies. Formerly Western Union Business Solutions and now privately held, the company is headquartered in Seattle and is described as the largest non‑bank provider in its market.