Senior Data Platform Architect (Snowflake, Databricks, Open Platforms & Data Virtualization), VP I - State Street Investment Management
State Street13 days ago
Base Salary
$120k - $203k/yr
Responsibilities
- Define and govern hybrid Snowflake–Databricks architectures for data warehousing, lakehouse, streaming, machine learning, analytics, and regulatory reporting.
- Establish enterprise reference architectures, lakehouse and medallion patterns, open table format standards, catalog federation, and interoperability across Snowflake, Databricks, Trino, Spark, Athena, and multiple clouds.
- Architect data virtualization and federated access across Snowflake, Databricks, Oracle, SQL Server, legacy NAS/file systems, Denodo, and Starburst while controlling performance, cost, security, and data duplication.
- Lead migration and modernization of batch applications, stored procedures, PL/SQL, mainframe workloads, Unix/Linux ETL, and file-based workflows to cloud-native platforms.
- Design batch ingestion, orchestration, monitoring, rerun safety, resiliency, reprocessing, transaction rollback, event logging, query limits, and warehouse credit controls.
- Design and enforce role-based access, row and column security, masking, secure views, network controls, cloud security integrations, lineage, audit trails, and regulatory data-protection controls.
- Architect multi-cloud integrations and cross-region or cross-cloud high availability and disaster recovery across AWS, Azure, and GCP.
- Design monitoring and chargeback models and identify outdated drivers, application versions, and expensive processes requiring remediation.
- Serve as a subject-matter expert during architecture reviews and escalations, partnering with Security, Risk, Audit, Application, Cloud Engineering, and other stakeholders.
- Produce architecture diagrams, design artifacts, executive presentations, access models, and reusable Center of Excellence assets.
Requirements
- Deep experience with Snowflake architecture, warehouse sizing, object hierarchy, Snowsight, RBAC, Horizon Catalog, governance, performance, data sharing, external tables, stages, loading, stored procedures, scripts, and Snowpark.
- Experience with Databricks, Apache Spark, Unity Catalog, ML pipelines, Delta Lake, Apache Iceberg, Lakehouse Federation, and lakehouse architectures.
- Experience with open data platforms and formats including Apache Iceberg, Delta Lake, Parquet, Polaris, Trino, and Starburst.
- Experience designing data virtualization, federated queries, logical data fabrics, and virtual data products using Denodo, Starburst, Snowflake federation, and Databricks federation.
- Experience modernizing batch applications and migrating Oracle, PL/SQL, mainframe, Unix ETL, and Control-M workloads to cloud-native orchestration.
- Experience with AWS, Azure, and GCP storage, IAM, networking, private connectivity, high availability, and disaster recovery.
- Knowledge of data modeling, dimensional modeling, denormalization, medallion schemas, bridge tables, marts, primary and foreign keys, clustering, and tuning implications.
- Programming and scripting experience with SQL, Python, Java, JavaScript, Scala, Spark, notebooks, and Snowpark.
- Strong Linux knowledge, including working with EC2 servers and command-line tools such as vi, cat, and more.
- Experience in regulated financial-services environments, including SOX, MRAs, audits, and NPPI/PII handling.
- Comfort with generative AI scripting and prompting, including using well-defined prompts and skill files to generate scripts.
- Strong communication, collaboration, knowledge-sharing, and stakeholder coordination skills.
Benefits
- Annual salary range of $120,000–$202,500, with the applicable range potentially differing outside the primary location.
- Retirement savings plan with company match and insurance coverage including life, medical, dental, vision, and long-term disability.
- Paid time off including vacation, sick leave, short-term disability, and family-care leave.
- Employee Assistance Program, incentive compensation eligibility, and certain tax-advantaged savings plans.
- Inclusive development opportunities, flexible work-life support, paid volunteer days, and employee networks.
Tech Stack
Apache AirflowApache SparkAWSAzureDatabricksGoogle CloudGoogle Cloud PlatformJavaJavaScriptLinuxMicrosoft SQL ServerPythonScalaSnowflakeSQL
Categories
Data Engineering
About State Street
State Street is a global custodian bank and asset manager serving institutional investors with investment servicing, fund accounting, trading/FX, data, and index/active strategies. Its products include State Street Global Advisors’ funds and ETFs, the Charles River Investment Management Solution, and the front-to-back State Street Alpha platform. Founded in 1792 and headquartered in Boston, the company is publicly traded on the NYSE (ticker: STT) and operates across major markets worldwide.