7 hours ago
Chennai, IndiaSenior
Responsibilities
- Design, develop, and optimize scalable ELT/ETL pipelines in Python and SQL.
- Build real-time, near-real-time, and batch data processing frameworks using cloud-native services.
- Implement incremental loads, CDC, SCD, schema evolution, and orchestration practices.
- Architect and manage Snowflake warehouses, databases, schemas, resource monitors, RBAC, zero-copy clones, clustering, and micro-partitioning.
- Implement Snowflake Tasks, Streams, Pipes, and External Tables for event-driven data workflows.
- Optimize Snowflake compute costs and query performance.
- Build and maintain Azure cloud solutions using Blob Storage, ADLS, Functions, ADF, and Purview.
- Design enterprise data warehouses, data marts, semantic layers, and modern data modeling patterns.
- Partner with BI and machine learning teams to operationalize features and analytics models.
- Design, develop, and deploy AI and generative AI solutions aligned with business needs.
- Implement CI/CD, DevSecOps controls, infrastructure-as-code, automated data testing, schema validation, data profiling, lineage, quality dashboards, and SLA monitoring.
- Lead design reviews, code reviews, and data platform roadmap discussions while mentoring junior engineers.
Requirements
- 6–10 years of professional experience in data engineering.
- Strong Python expertise, including pandas, asyncio, object-oriented programming, typing, packaging, and pytest.
- Hands-on enterprise-scale Snowflake experience and advanced SQL skills.
- Strong experience with at least one major cloud platform: AWS, Azure, or GCP.
- Deep understanding of data warehousing, dimensional modeling, star schemas, data marts, CDC/SCD, and high-volume pipelines.
- Experience with GitHub Actions, Azure DevOps, or GitLab CI; Terraform or Bicep; and DevSecOps practices.
- Experience with orchestration tools such as Airflow, ADF, Prefect, or Step Functions.
- Experience with dbt, streaming platforms, data lineage tools, and lakehouse architectures.
- Understanding of MLOps, feature stores, cost governance, and FinOps.
- Prior leadership or mentorship experience and strong communication and stakeholder management skills.
Benefits
- Flexible working environment.
- Volunteer time off.
- LinkedIn Learning.
- Employee Assistance Program (EAP).
Tech Stack
Apache AirflowApache KafkaAWSAzuredbtGitHub ActionsGitLab CI/CDGoogle Cloud PlatformPandaspytestPythonSnowflakeSQLTerraform
Categories
Data Engineering