Databricks Platform Engineer - Technology Services Section, Global IT Strategy Office (GITSO)
Rakuten24 hours ago
Tokyo, JapanSenior
Responsibilities
- Design and implement a contract-driven Declarative Automation Bundle generator that translates Open Data Contract Standard YAML files into deployable Databricks assets.
- Build and maintain deployment pipelines with environment parameterization, atomic deployment, rollback, approvals, variable groups, and service connections.
- Develop Azure Pipelines CI/CD workflows and enforce Git branching, merge, and environment-promotion policies.
- Automate Unity Catalog schema, table, tag, grant, column-mask, and row-filter management through the Databricks SDK, REST APIs, and SQL DDL.
- Design and maintain the contract index, dead-letter asset pattern, schema validation, smoke tests, and idempotent deployment verification.
- Build platform health dashboards, alerting, freshness signals, SLA/SLO comparisons, dead-letter monitoring, operational reports, and cost attribution.
- Implement monitoring for SCIM synchronization latency, deployment-ordering failures, break-glass activation, and data quality constraints.
- Contribute to Databricks workspace deployment and administration with Cloud Platform Engineers.
Requirements
- At least 5 years of experience in data engineering, data platform engineering, or DataOps.
- At least 3 years of experience with Databricks, including Unity Catalog, workspace administration, the Databricks SDK, and REST APIs.
- Strong Python proficiency for automation scripting, YAML parsing, code generation, and API integration.
- Solid software engineering practices covering modular design, maintainability, error handling, testing, and documentation.
- Experience with Declarative Automation Bundles or equivalent infrastructure-as-code deployment frameworks for Databricks.
- Understanding of RBAC and governance models, including schemas, grants, column masks, and row filters.
- Experience with CI/CD pipelines, particularly Azure DevOps and Azure Pipelines, including multi-stage YAML pipelines, environment approvals, and service connections.
- Experience building monitoring dashboards, alerting systems, and operational reporting.
- SQL proficiency is desired.
- Familiarity with the Open Data Contract Standard or similar data contract frameworks is desired.
- Experience with Delta Lake internals, data quality frameworks, SLA/SLO monitoring, Git workflow design, Azure Entra ID, SCIM provisioning, or workload identity federation is desired.
- English at business level is required; Japanese is preferred.
- Experience in IT operations domains such as license management or asset management is desired.
About Rakuten
Rakuten Group builds e-commerce marketplaces, financial services (credit cards, payments, banking, securities), and digital/communications offerings for consumers and merchants worldwide. It monetizes through transaction fees, financial services income, advertising, and mobile subscriptions; flagship businesses include Rakuten Ichiba, Rakuten Card, and Rakuten Mobile. Founded in 1997 and headquartered in Setagaya, Tokyo, the company is publicly listed on the Tokyo Stock Exchange and operates across many countries and regions.