Sr. Designated Support Engineer, Apache Spark™
Databricks2 months ago
Amsterdam, NetherlandsSenior
Responsibilities
- Manage large strategic accounts and provide a high-touch, proactive support experience.
- Troubleshoot complex Apache Spark and data engineering issues, including performance bottlenecks, data skew, out-of-memory errors, serialization problems, and slow broadcast joins.
- Analyze Spark UI, thread dumps, and driver/executor logs to diagnose and resolve critical issues.
- Advise customers on data engineering best practices, ETL/ELT pipelines, data modeling, and Databricks Lakehouse architecture.
- Project manage open support cases and coordinate resolution across Engineering, Product, subject-matter experts, and account teams.
- Engage senior customer and internal stakeholders to remove roadblocks, mitigate issues, and restore customer functionality.
- Review customer cases to identify trends and proactive optimization opportunities.
- Champion customer needs, chair stakeholder roundtables, and promote customer self-sufficiency with Databricks Support.
Requirements
- At least 3 years of dedicated experience building, deploying, and maintaining large-scale data pipelines in data engineering.
- At least 3 years of experience designing, developing, testing, and sustaining applications based on Python, Java, or Scala.
- Deep architectural knowledge of Apache Spark internals, including drivers, executors, memory management, DAGs, Catalyst Optimizer, and the Tungsten execution engine.
- Proven experience troubleshooting complex Spark and big data issues.
- Strong project management, analytical, problem-solving, written, and verbal communication skills.
- Ability to translate complex technical concepts for non-technical stakeholders and demonstrate executive presence.
- Customer-facing experience as a Support Engineer, Technical Account Manager, Designated Support Engineer, or in related production support is preferred.
Benefits
- Databricks offers comprehensive benefits and perks tailored to employees' needs, with specific details varying by region.
Tech Stack
Categories
Solutions Engineering
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.