Senior Software Engineer, Data & Platform Services
LivePerson2 months ago
Responsibilities
- Lead the migration and re-architecture of mission-critical data pipelines from Hadoop, Spark, and Impala to Databricks.
- Convert complex SQL, modernize ETL scripts, and migrate integrations, connections, and secrets.
- Use Claude, Codex, and similar AI coding assistants to accelerate code conversion and modernization while validating architecture, performance, quality, and maintainability.
- Design, develop, debug, and optimize scalable data workflows using Airflow and Databricks.
- Implement data validation and testing strategies to preserve accuracy and consistency during migrations.
- Improve observability, monitoring, incident response, SLOs, SLAs, and production on-call practices.
- Build data platform capabilities supporting contact center analytics, conversation insights, reporting, and business intelligence.
Requirements
- 6–8 years of experience building and supporting distributed systems and large-scale data platforms.
- Experience leading or substantially contributing to enterprise data platform migrations from Hadoop, Spark, Hive, or Impala to cloud-based platforms such as Databricks.
- Strong Databricks expertise, including Jobs, Delta Lake, cluster configuration and optimization, and migrating Impala or Hive SQL workloads to Spark SQL.
- Advanced Python programming skills for ETL development and extensive experience with production data pipelines and Airflow orchestration.
- Experience with GCP and GCS, containerized environments, Kubernetes, and Kubernetes-based GKE architectures.
- Understanding of distributed data processing, real-time and batch processing trade-offs, Hadoop, MapReduce, data integration patterns, and event-driven architectures.
- Experience building reliable and observable platforms with monitoring, validation, and data quality controls.
- Experience using AI-powered coding tools such as Claude, Codex, or similar assistants, including reviewing and validating generated code.
- Strong communication and collaboration skills in globally distributed, cross-functional engineering teams.
- Experience with conversational AI, contact center platforms, or customer engagement technologies is preferred.
- Proficiency in Java or Scala is preferred.
- Experience designing and maintaining CI/CD pipelines is preferred.
Benefits
- Fully remote, remote-first work model with no defined working hours or boundaries.
- 33 days of personal time off, including 28 vacation days and 5 care days.
- Paid public holidays and volunteering days.
- Additional pension scheme with a 20% LivePerson contribution, ESPP, and performance-based annual bonus.
- Internet and mobile reimbursement.
- Access to internal professional development resources.
- Monthly office connection pizza days in Mannheim or Berlin.
Tech Stack
Apache AirflowApache HadoopApache HiveApache SparkDatabricksGoogle CloudGoogle Cloud PlatformJavaKubernetesPythonScala
Categories
Data EngineeringDevOps
About LivePerson
LivePerson (NASDAQ: LPSN) is the leader in predictable conversational AI and digital transformation. The world's leading brands use our award-winning Conversational Cloud platform to connect with millions of consumers. We power nearly a billion conversational interactions every month, providing uniquely rich data analytics and safety tools to unlock the power of conversational AI for better business outcomes. Fast Company named LivePerson the #1 Most Innovative AI Company in the world. Learn more at liveperson.com.