11 hours ago
Chennai, IndiaSenior
Responsibilities
- Design, develop, and optimize scalable ELT/ETL pipelines using Python and SQL.
- Build real-time, near-real-time, and batch data frameworks using cloud-native services.
- Implement incremental loads, CDC, SCD, schema evolution, and orchestration best practices.
- Architect and manage enterprise-scale Snowflake environments, including warehouses, databases, schemas, resource monitors, RBAC, zero-copy clones, clusters, and micro-partitioning.
- Implement Snowflake Tasks, Streams, Pipes/Snowpipe, and External Tables for event-driven data workflows.
- Optimize Snowflake compute costs and query performance through clustering, micro-partitioning, caching, and warehouse sizing.
- Build and maintain Azure cloud-native compute and storage solutions.
- Design enterprise data warehouses, data marts, and semantic layers using Kimball, Data Vault, and modern ELT-first patterns.
- Design, develop, and deploy AI and Generative AI solutions aligned with business requirements.
- Implement CI/CD pipelines, DevSecOps controls, and infrastructure using infrastructure-as-code tools.
- Implement automated data unit testing, schema validation, contract enforcement, profiling, quality dashboards, lineage, and SLA monitoring.
- Lead design reviews, code reviews, and data platform roadmap discussions.
- Mentor junior engineers and collaborate with Product, Data Science, BI, and Business teams.
Requirements
- 6–10 years of professional experience in Data Engineering.
- Strong expertise in Python, including pandas, asyncio, object-oriented programming, typing, packaging, and pytest.
- Hands-on experience with Snowflake at enterprise scale.
- Advanced SQL skills, including complex joins, window functions, common table expressions, and performance tuning.
- Strong cloud background with AWS, Azure, or GCP.
- Deep understanding of data warehousing, dimensional modeling, star schemas, data marts, CDC/SCD, and high-volume pipeline modeling.
- Experience with CI/CD using GitHub Actions, Azure DevOps, or GitLab CI.
- Experience with infrastructure-as-code tools such as Terraform and Bicep.
- Strong understanding of DevSecOps, secrets management, IAM design, vulnerability scanning, and zero-trust principles.
- Experience with orchestration tools such as Airflow, ADF, Prefect, or Step Functions.
- Experience with dbt models, tests, and exposures.
- Experience with streaming platforms such as Kafka, Kinesis, or Pub/Sub.
- Experience implementing OpenLineage, Marquez, or other lineage tools.
- Familiarity with Lakehouse architectures including Delta Lake, Iceberg, or Hudi.
- Understanding of MLOps concepts and feature stores.
- Knowledge of cost governance and FinOps best practices.
- Prior leadership and mentorship experience.
- Strong communication and stakeholder-management skills.
Benefits
- Flexible working environment.
- Volunteer time off.
- LinkedIn Learning.
- Employee Assistance Program (EAP).
Tech Stack
Apache AirflowApache KafkaAWSAzuredbtGitHub ActionsGitLab CI/CDGoogle Cloud PlatformPandaspytestPythonSnowflakeSQLTerraform
Categories
Data Engineering