Data Engineer
Sumitomo Mitsui Banking Corporation3 months ago
Base Salary
$73k - $98k/yr
Responsibilities
- Design and implement Databricks Medallion Architecture pipelines for structured and unstructured data.
- Develop, scale, and orchestrate workflows using Databricks Jobs and Delta Live Tables.
- Automate deployment of Databricks assets, workspace configurations, and code pipelines across development, QA, and production environments.
- Integrate Databricks with Amazon S3, AWS Glue, and AWS IAM for cataloging, storage, data movement, and secure access control.
- Tune Spark clusters, optimize Delta Lake storage through techniques such as Z-Ordering and partitioning, and manage AWS compute costs.
- Implement fine-grained data access controls, lineage, and auditing using Unity Catalog or native cloud security controls.
- Build production-grade ETL/ELT pipelines with schema validation and cleansing of raw capture feeds.
- Partner with data scientists, risk managers, and downstream analytics teams to provide business-ready data views for reporting and AI modeling.
Requirements
- Professional experience in architectural design and development on the Databricks platform in an AWS cloud environment.
- Proficiency with Databricks Asset Bundles, Terraform using the Databricks and AWS providers, and Git-based deployment pipelines.
- Programming experience with Python, PySpark, and SQL for complex data transformation and manipulation.
- Strong understanding of Apache Spark internals, Delta Lake mechanics, and streaming data concepts.
- Experience working with Amazon MSK or Kafka data streams.
- Experience building production-grade ETL/ELT pipelines, validating schemas, and cleansing raw data feeds.
- Databricks Certified Data Engineer Professional or AWS Certified Data Engineer – Professional certification is highly advantageous.
Benefits
- Anticipated base salary is $73,000.00 to $98,000.00, with potential eligibility for an annual discretionary incentive award.
- SMBC offers a competitive portfolio of employee benefits.
- Hybrid work is available, combining work from home with work in an SMBC office.
- Employees must live within a reasonable commuting distance of their office; the specific hybrid schedule is provided during the interview process.
- Reasonable accommodations are available during candidacy for applicants with disabilities.
Tech Stack
Categories
Data Engineering
About Sumitomo Mitsui Banking Corporation
Sumitomo Mitsui Banking Corporation provides corporate, investment, and commercial banking services—such as lending, capital markets, global trade finance, treasury/FX, and transaction banking—to corporate, institutional, and consumer clients. Headquartered in Tokyo, it is the core banking subsidiary of Sumitomo Mitsui Financial Group (SMFG), one of Japan’s three major banking groups, whose shares trade in Tokyo and as NYSE ADRs (SMFG). SMBC serves clients through a network spanning nearly 40 countries.