5 hours ago
Noida, IndiaSenior
Responsibilities
- Design source control workflows, branching models, environment provisioning, and release management strategies.
- Build, maintain, and optimize multi-stage CI/CD pipelines with GitHub Actions and Azure Pipelines.
- Automate cloud infrastructure provisioning with Terraform modules, state management, remote backends, and GitHub workflows.
- Automate Azure Databricks workspace, cluster, pool, and storage-mount deployment using the Databricks Terraform Provider.
- Create CI/CD pipelines for Databricks workflows and automate deployment of notebooks, Delta Live Tables, and jobs across development, test, and production environments.
- Manage Azure services including Azure Kubernetes Service, Azure Container Registry, and serverless functions.
- Administer Git repositories and enforce branch protection, code ownership, and pull-request workflows.
- Integrate security scanning and governance controls, including GitHub Advanced Security, Dependabot, Secret Scanning, Azure Policy, and Databricks Secret Scopes.
- Set up observability, logging, and telemetry with Azure Monitor and Databricks diagnostic logs.
- Collaborate with data engineers, developers, QA testers, clients, and leadership while tracking sprints and release milestones.
Requirements
- At least 5 years of experience, preferably including foundational Data Engineering experience.
- Strong experience with GitHub, GitHub Actions, GitHub Advanced Security, GitHub Enterprise features, and Azure DevOps.
- Advanced hands-on Terraform expertise, including modules, state management, workspace provisioning, and remote backends.
- Experience with Azure Databricks, including the CLI, REST API, workspace management, and Unity Catalog.
- Experience with Microsoft Azure compute, networking, IAM, and Azure Key Vault.
- Scripting and automation experience with PowerShell, Bash, or Python.
- Comfort working across AWS, Azure, and GCP, with a preference for Azure.
- Ability to independently manage projects end to end and collaborate frequently with clients and leadership.
- Excellent communication and presentation skills with confidence interacting with stakeholders.
- Flexibility to work partially in EST time zones.
Benefits
- Competitive salary and a strong insurance package.
- Extensive employee learning and development resources.
Tech Stack
Categories
Data EngineeringDevOps
About ShyftLabs
ShyftLabs builds and runs enterprise data and AI platforms, combining data engineering, analytics, and responsible AI controls with consulting and managed delivery. It designs governed data infrastructure, customer and retail analytics, and omnichannel marketing data products for large organizations. Founded in 2018 and headquartered in Toronto, the privately held firm serves global clients and staffs teams across Toronto, New York, Dubai, and New Delhi.
