Sr. Databricks Data Engineer, Onsite AVP
State Street24 hours ago
Toronto, CanadaSenior
Base Salary
$91k - $141k/yr
Responsibilities
- Design and develop high-throughput, configurable data pipelines using Databricks.
- Build and test end-to-end streaming, event-driven, batch, ingestion, integration, and data curation pipelines.
- Develop and deploy scalable ETL pipelines using Databricks SQL and Scala/PySpark.
- Optimize Databricks job performance through query optimization, resource management, and scaling strategies.
- Troubleshoot data, environment, DevOps pipeline, and Azure service issues, including root-cause and impact analysis.
- Participate in all stages of the software development lifecycle, including design, coding, integration testing, deployment, and documentation.
- Collaborate with team members, users, and geographically distributed stakeholders to drive technical change.
Requirements
- Bachelor’s degree in computer science, information technology, or a related subject.
- 7+ years of overall IT experience.
- 5+ years of data engineering experience handling large datasets.
- 3+ years of hands-on Databricks experience using SQL and Scala/PySpark.
- 3+ years of cloud-based development experience using Azure or AWS.
- Strong hands-on experience with Databricks SQL, Scala/PySpark, data engineering pipelines, DevOps pipeline troubleshooting, and Azure services.
- Experience building data lakehouse solutions with Databricks and working with microservices and API- or event-driven architecture.
- Experience in the financial industry and with agile development methodology.
- Strong communication, critical-thinking, problem-solving, collaboration, multitasking, initiative, and professional work-ethic skills.
Benefits
- Onsite work four days per week with one remote day.
- Standard business hours are generally 8:00 a.m. to 5:00 p.m., aligned to project and stakeholder needs.
- Inclusive development opportunities, flexible work-life support, paid volunteer days, and employee networks.
About State Street
State Street is a global custodian bank and asset manager serving institutional investors with investment servicing, fund accounting, trading/FX, data, and index/active strategies. Its products include State Street Global Advisors’ funds and ETFs, the Charles River Investment Management Solution, and the front-to-back State Street Alpha platform. Founded in 1792 and headquartered in Boston, the company is publicly traded on the NYSE (ticker: STT) and operates across major markets worldwide.