1 day ago
Lisbon, PortugalMid Level / Senior / Staff+
H1B Sponsor
Responsibilities
- Design, build, and maintain scalable data engineering frameworks and platform utilities used across engineering teams.
- Develop reusable patterns, templates, and abstractions to standardize and accelerate delivery.
- Define and evolve platform architecture decisions, ensuring scalability, maintainability, and consistency.
- Design and implement CI/CD pipelines and automation frameworks to improve engineering velocity.
- Define and enforce engineering standards for testing, code quality, deployment, and documentation.
- Identify and eliminate manual or repetitive processes through automation and tooling improvements.
- Integrate AI-assisted development tools into engineering workflows to improve productivity.
- Lead the development and operational support of core data transformation frameworks.
- Investigate and resolve framework-level issues, including deployment failures and production incidents.
- Support onboarding and enablement of engineering teams adopting platform tooling.
Requirements
- Strong proficiency in Python for building frameworks, automation tooling, and reusable components.
- Hands-on experience with Databricks, including notebooks, workflows, jobs, and Unity Catalog.
- Strong SQL skills and experience with distributed processing frameworks such as Apache Spark.
- Deep experience with dbt Core, including project structure, models, tests, macros, and deployment at scale.
- Proven experience designing and maintaining CI/CD pipelines using tools like GitHub Actions or Azure DevOps.
- Experience with data engineering platform design, including scalable pipeline and workflow architectures.
- Strong understanding of software engineering principles such as DRY, SOLID, and modular design.
- Experience working with version control systems and modern Git workflows.
- Experience with cloud platforms, preferably AWS, and infrastructure-as-code concepts like Terraform.
- Experience implementing automated testing strategies (unit, integration, data quality).
Benefits
- Flexibility, with remote and hybrid work options.
- Career advancement opportunities, including international mobility and professional development programs.
- Access to cutting-edge tools, training, and industry experts for learning and development.