2 months ago
Tel Aviv-Yafo, IsraelSenior
Responsibilities
- Design, build, and own production-grade data pipelines using Python and SQL.
- Develop stateless and idempotent pipelines resilient to retries, failures, and infrastructure interruptions.
- Implement data transformations, validation, and data quality checks.
- Optimize pipelines for performance, reliability, and cost efficiency.
- Collaborate with Analytics, Data Science, and ML teams to deliver trusted datasets.
- Orchestrate pipelines using Airflow or an equivalent workflow orchestration framework.
- Package workloads with Docker and deploy them on Kubernetes.
- Build CI/CD automation and provision required pipeline infrastructure with Infrastructure as Code.
Requirements
- At least 8 years of experience as a Data Engineer focused primarily on building data pipelines.
- At least 6 years of hands-on experience with Python and SQL.
- At least 3 years of experience running workloads on Kubernetes.
- Strong understanding of stateless system design and idempotent data processing.
- Experience operating data pipelines in cloud environments and using workflow orchestration frameworks.
- Strong Linux fundamentals and production debugging skills.
- Working knowledge of spoken and written English.
- Preferred experience with open-source software, Apache Spark or similar distributed processing frameworks, machine learning data pipelines, cost-optimized compute, relational and non-relational data stores, and large-scale or high-reliability data systems.
Benefits
- Competitive compensation.
- Career growth and learning opportunities.
- Flexibility and ownership.
- Collaborative and innovative culture.
- Opportunity to work on impactful AI projects.
- International environment and talented teams.
- Applicants must be authorized to work in the country in which they apply and provide proof of employment eligibility.
- The role welcomes employees to work from Nebius's offices in Tel Aviv, Israel.
Tech Stack
Categories
Data Engineering
About Nebius
Nebius builds a full-stack AI cloud offering GPU compute, storage, and tools for training and deploying ML models for startups, enterprises, and research labs. It sells consumption-based cloud infrastructure (IaaS/PaaS) and managed services tailored to generative AI workloads, including large-scale model training and inference. The company is headquartered in Amsterdam and operates as an independent provider.