Senior Data Engineer (Databricks Migration)
Sigma Software21 days ago
Remote, Romania or Bucharest, RomaniaSenior
Responsibilities
- Migrate large-scale analytical and ETL/ELT workloads from BigQuery to Databricks.
- Design and implement scalable Lakehouse architectures using Databricks and Delta Lake.
- Develop, optimize, and maintain Python, SQL, Spark, and PySpark data pipelines.
- Implement incremental processing strategies, scalable transformation frameworks, and Bronze, Silver, and Gold medallion layers.
- Apply data governance and security best practices using Unity Catalog.
- Design reusable data platform components and contribute to engineering standards and architecture decisions.
- Troubleshoot and optimize complex SQL and Spark workloads, including cluster monitoring, metrics analysis, and performance optimization.
- Support production deployments, platform reliability, maintainability, and modernization activities.
- Collaborate with Data Science, Analytics, Product, and Customer Engineering teams in international cross-functional environments.
- Conduct code reviews and contribute to CI/CD practices for data engineering platforms.
Requirements
- At least 5 years of professional experience as a Data Engineer.
- Strong programming skills in Python and advanced SQL.
- Hands-on commercial experience with Databricks, Delta Lake, Apache Spark, and primarily PySpark.
- Experience designing modern cloud-based data platforms and large-scale ETL/ELT processing solutions.
- Experience with Spark Declarative Pipelines, cluster monitoring, metrics analysis, and performance optimization.
- Strong understanding of distributed data processing architectures, data warehousing, and dimensional modeling.
- Experience with Airflow or similar orchestration tools.
- Experience optimizing complex analytical SQL workloads and implementing CI/CD practices for data engineering platforms.
- Strong troubleshooting, analytical, problem-solving, communication, and collaboration skills.
- Upper-Intermediate or higher English proficiency.
- Preferred: experience with GCP, AWS, or Azure; retail analytics or pricing optimization; machine learning or AI-related data workloads; and platform modernization or cloud migration initiatives.
Benefits
- Opportunities for continuous learning and technology growth.
- Work on complex international projects with meaningful engineering impact.
- Collaboration with international cross-functional teams on an enterprise-scale retail analytics platform.
Tech Stack
Categories
Data Engineering
About Sigma Software
Sigma Software is a London-headquartered, privately held IT consultancy that designs and builds custom software for startups and enterprises. The company provides end-to-end services—engineering, web and mobile development, .NET/Java platforms, AR/VR, and UI/UX—delivered via outsourcing, managed teams, and consulting. Founded in 2002, it supports long-term product development and modernization projects across multiple industries for global clients.