Airflow Platform Engineer
Jade Global4 months ago
Pune, India or Hyderābād, IndiaMid Level / Senior
Responsibilities
- Design, develop, optimize, troubleshoot, and maintain enterprise Apache Airflow DAGs using advanced orchestration patterns.
- Enable modern data workflows, including DBT pipelines, and provide reusable templates, best practices, and onboarding support.
- Operate and support Airflow deployed on AKS, including its scheduler, webserver/API, workers, and metadata services.
- Diagnose Kubernetes issues involving pods, networking, storage, and RBAC, as well as Airflow scheduling and resource problems.
- Support CI/CD pipelines for DAG and container deployments and improve platform stability, monitoring, and alerting.
- Develop and maintain unit, integration, and DAG validation tests, including operator mocking and dependency validation.
Requirements
- 4–6 years of experience.
- Strong production experience with Apache Airflow 3.x or later and Airflow DAG development.
- Working knowledge of Kubernetes, preferably AKS, for supporting the Airflow runtime.
- Expert-level Python skills for building, debugging, and maintaining production workflows and supporting libraries.
- Expertise writing unit, integration, and DAG validation tests for Airflow workflows in CI/CD environments.
- Experience with DBT, modern data pipelines, and CI/CD tools including Git, Jenkins, and Docker.
- Familiarity with Azure or other cloud platforms.
- Preferred: experience with Astronomer/Astro Private Cloud, Snowflake, APIs, Splunk, Prometheus, and Azure Key Vault.
Categories
Data EngineeringDevOps
About Jade Global
Jade Global provides IT consulting, systems integration, and managed services for enterprises, covering cloud transformation, ERP/CRM modernization, data and analytics, integration, and automation. It implements and supports platforms from Oracle, Salesforce, SAP, ServiceNow, Workday, Snowflake, and Boomi. Privately held and founded in 2003, the company is headquartered in San Jose, California and serves customers across North America, Europe, and APAC.