Data Engineer - Capital Markets
Sumitomo Mitsui Banking Corporation3 months ago
Base Salary
$97k - $156k/yr
Responsibilities
- Design and implement structured and unstructured data pipelines using Databricks Medallion Architecture across Bronze, Silver, and Gold layers.
- Develop, scale, and orchestrate workflows with Databricks Jobs and Delta Live Tables.
- Automate deployment of Databricks assets, workspace configurations, and code pipelines across Dev, QA, and Production environments.
- Integrate Databricks with Amazon S3, AWS Glue, and AWS IAM for cataloging, storage, data movement, and secure access control.
- Tune Spark clusters, optimize Delta Lake storage through Z-Ordering and partitioning, and manage AWS compute costs.
- Implement fine-grained access controls, data lineage, and auditing using Unity Catalog or native cloud security controls.
- Build production-grade ETL/ELT pipelines with schema validation and cleansing of raw capture feeds.
- Partner with Data Scientists, Risk Managers, and downstream analytics teams to deliver business-ready data views for reporting and AI modeling.
- Support real-time financial trading data consumption, regulatory reporting, real-time reporting, and advanced analytics environments.
Requirements
- At least 4 years of hands-on experience in architectural design and development within the Databricks platform.
- At least 3 years of experience working in an AWS cloud environment.
- Proficiency with Databricks Asset Bundles, Terraform Databricks and AWS providers, and Git-based deployment pipelines such as GitHub Actions, GitLab CI/CD, or AWS CodePipeline.
- Advanced programming skills in Python, PySpark, and SQL for complex data manipulation and transformation.
- Strong understanding of Apache Spark internals, Delta Lake mechanics, and streaming data concepts, including Amazon MSK or Kafka data streams.
- Proven experience building production-grade ETL/ELT pipelines, validating schemas, and cleansing raw capture feeds.
- Databricks Certified Data Engineer Professional or AWS Certified Data Engineer – Professional certification is highly advantageous.
Benefits
- The anticipated annual salary range is $97,000.00 to $156,000.00, with potential eligibility for an annual discretionary incentive award.
- SMBC offers a competitive portfolio of employee benefits.
- Hybrid work is available from home and an SMBC office, and employees must live within a reasonable commuting distance of their office.
- The specific hybrid schedule is provided during the interview process, and some roles may require full-week in-office attendance.
- Reasonable accommodations are available during candidacy for applicants with disabilities.
Tech Stack
Categories
Data Engineering
About Sumitomo Mitsui Banking Corporation
Sumitomo Mitsui Banking Corporation provides corporate, investment, and commercial banking services—such as lending, capital markets, global trade finance, treasury/FX, and transaction banking—to corporate, institutional, and consumer clients. Headquartered in Tokyo, it is the core banking subsidiary of Sumitomo Mitsui Financial Group (SMFG), one of Japan’s three major banking groups, whose shares trade in Tokyo and as NYSE ADRs (SMFG). SMBC serves clients through a network spanning nearly 40 countries.