Data Platform Engineer
Greystar Real Estate Services, LLC19 days ago
Remote, United StatesSenior
Base Salary
$125k - $150k/yr
Responsibilities
- Administer Databricks workspaces, cluster policies, compute configurations, upgrades, Unity Catalog permissions, service principals, groups, users, and governance policies.
- Tune Spark and Delta Lake workloads for performance, reliability, and cost through partitioning, Z-ordering, VACUUM, OPTIMIZE, compute controls, and transaction-log management.
- Operate the Azure data platform, including ADLS Gen2, Azure Data Factory, Azure Monitor, Log Analytics, Key Vault, Event Hub, Synapse Analytics, Cosmos DB, and Azure SQL Server.
- Define and enforce platform standards for naming, schema evolution, medallion architecture, data contracts, SLA tiers, environment parity, and pipeline design.
- Own CI/CD and environment promotion for data workflows using GitHub Enterprise, Azure DevOps Pipelines, automated testing gates, rollback procedures, and release controls.
- Manage Azure Active Directory identities, service principals, RBAC assignments, Unity Catalog access policies, data lineage, metadata, audit readiness, and compliance documentation.
- Maintain Terraform and other infrastructure-as-code configurations for reproducible Databricks and Azure environment provisioning.
- Build AI-powered platform observability and self-healing capabilities for anomaly detection, pipeline drift, SLA-risk prediction, incident summarization, automated retries, circuit breakers, and remediation.
- Design and maintain disaster recovery and business continuity strategies, including backup policies, geo-redundancy, workspace recovery, RTO/RPO targets, and failover testing.
- Participate in on-call coverage, manage ServiceNow and Asana ticket queues, track Linear and DevOps work items, and produce runbooks, deployment playbooks, post-incident reviews, ADRs, and platform specifications.
Requirements
- At least 5 years of platform engineering, infrastructure engineering, or production data operations experience.
- Expert Databricks administration experience covering Unity Catalog, workspace management, cluster policies, compute configuration, and Delta Lake internals.
- Strong experience with Azure data services including ADLS Gen2, Azure Data Factory, Azure Monitor, Log Analytics, Key Vault, Azure Cost Management, and Event Hub.
- Experience with Azure Active Directory, service principals, RBAC, Unity Catalog governance, Azure networking, VNets, private endpoints, NSGs, firewalls, and DNS.
- Hands-on experience designing and orchestrating Azure Data Factory pipelines at scale.
- Experience with Spark performance tuning, partitioning, Z-ordering, compute-cost optimization, and bottleneck profiling.
- Proven GitHub Enterprise CI/CD experience for environment promotion, rollback, and data-pipeline release management.
- Python and/or SQL experience for automation, compliance validation, and platform tooling.
- Knowledge of medallion and lakehouse architecture patterns and disciplined multi-environment deployment practices.
- Terraform and infrastructure-as-code experience provisioning Databricks workspaces, Azure resources, Unity Catalog configurations, and environment-specific infrastructure.
- Experience authoring Azure DevOps Pipelines YAML, configuring build and release pipelines, and integrating Azure DevOps with GitHub Enterprise.
- Experience with Azure Monitor and Log Analytics, including KQL queries, alert rules, diagnostic settings, workbooks, and cluster-event monitoring.
- Knowledge of Delta Lake features including VACUUM, OPTIMIZE, ZORDER, time travel, table statistics, and transaction-log management.
- Preferred experience with Synapse Analytics, Cosmos DB, Azure SQL Server, Databricks Mosaic AI, Genie, data lineage, metadata management, Unity Catalog governance, Yardi, Entrata, RealPage, legacy BI migration, platform modernization, or Linear.
Benefits
- Competitive base salary of $125,000-$150,000 annually, with potential bonus eligibility; compensation figures are excluded from benefit details.
- Medical, dental, vision, disability, and life insurance, including low or free basic employee-only medical coverage.
- Paid time off including 15 vacation days, 4 personal days, 10 sick days, and 11 paid holidays for new hires, with additional vacation based on tenure.
- Birthday paid time off after one year of service.
- Flexible work arrangements; the position may be performed remotely anywhere in the United States except Alaska.
- Onsite housing discounts at Greystar-managed communities, subject to availability and applicable conditions.
- Six-week paid sabbatical after 10 years of service and every five years thereafter.
- 401(k) company match of up to 6% of pay after six months of service.
- Paid parental leave and lifetime fertility benefit reimbursement of up to $10,000, including adoption and surrogacy.
- Employee Assistance Program, critical illness, accident, hospital indemnity, pet insurance, legal plans, and charitable giving benefits.
- Benefits are offered to full-time employees and may vary for union or prevailing-wage roles.