21 days ago
Remote, United StatesSenior
Responsibilities
- Design, develop, and maintain PySpark and SQL data pipelines for clinical research data.
- Develop and maintain clinical and research data models in Databricks.
- Design and optimize data warehouse schemas and data models for large-scale datasets.
- Modernize on-premises Microsoft SQL Server ETL processes into cloud-based Databricks pipelines and workflows.
- Implement and monitor data quality checks, validation processes, logging, and pipeline performance.
- Troubleshoot and optimize Spark, SQL, and Databricks compute workloads.
- Participate in architecture discussions and document technical approaches and solutions.
- Perform coding, code reviews, testing, debugging, technical documentation, go-live planning, and post-live support.
- Collaborate with team members and research stakeholders to resolve issues and improve system performance and code quality.
- Train and mentor staff members and provide development documentation and instructions.
Requirements
- At least 10 years of related work experience.
- High school diploma or GED required.
- Technology-related bachelor's degree preferred.
- Proven experience with Databricks and cloud-based data engineering platforms.
- Strong SQL and Python/PySpark experience for developing and optimizing data processing pipelines.
- Experience with Apache Spark or other distributed data processing technologies.
- Experience designing data models and data warehouse schemas for large-scale datasets.
- Experience developing, maintaining, or modernizing ETL processes and data pipelines.
- Experience with relational databases and SQL Server.
- Experience migrating or modernizing on-premises ETL processes is preferred.
- Familiarity with AWS cloud services and cloud-based data architectures.
- Experience with data quality, validation, monitoring, logging, troubleshooting, scalable data processing, and performance optimization.
- Strong communication, critical-thinking, interpersonal, collaboration, and leadership skills.
Benefits
- Work from home in Pennsylvania, with a regular full-time schedule of 40 hours per week on day shifts.
- Healthcare benefits, including vision, dental, and domestic-partner coverage, are available from day one for full-time and part-time positions.
- Collaborative, cooperative, and collegial work environment.
Tech Stack
Categories
Data Engineering
About Geisinger
Geisinger is a nonprofit integrated health system in Pennsylvania that operates hospitals, clinics, a physician network, and the Geisinger Health Plan for patients and employers. It also runs Geisinger College of Health Sciences and conducts clinical research to support care across urban and rural communities. Founded in 1915 and headquartered in Danville, Pa., Geisinger became the first member of Risant Health in 2024.