DataOps Engineer

19 days ago

Remote, United StatesSenior

Base Salary

$120k - $150k/yr

Responsibilities

Implement AI-powered observability for pipeline drift, anomaly classification, SLA-risk prediction, and automated incident summaries.
Build agentic monitoring and self-healing workflows for data quality degradation, pipeline dropout, schema drift, and volume anomalies.
Operate Azure data services including ADLS Gen2, Azure Data Factory, Azure Monitor, Log Analytics, Key Vault, and Event Hub.
Design and maintain scalable Azure Data Factory ingestion and orchestration pipelines for multi-tenant ERP environments.
Design, build, optimize, and administer Databricks pipelines, workspaces, Unity Catalog, cluster policies, access controls, compute configurations, and Delta table lifecycles.
Tune Spark jobs for performance, reliability, partitioning, Z-ordering, and cost efficiency.
Own CI/CD and environment promotion across development, staging, and production using GitHub Enterprise and Linear.
Enforce deployment standards including testing gates, rollback procedures, change documentation, and environment parity controls.
Instrument data quality checks across Bronze, Silver, and Gold layers and enforce data contracts, ownership standards, and quality SLAs.
Produce runbooks, deployment playbooks, incident post-mortems, ADRs, and platform specifications.
Participate in on-call rotation and support SLA commitments for business-critical data domains.

7+ years of production experience in DataOps, data engineering, or platform engineering.
Expert hands-on experience with Databricks Delta Live Tables, Jobs and Workflows, Unity Catalog, Spark performance tuning, and Delta Lake internals.
Strong command of Azure data services including Azure Data Factory, ADLS Gen2, Azure Monitor, Log Analytics, and Key Vault.
Production experience using AI tools for DataOps or data observability, such as LLM-assisted diagnostics, intelligent alerting, or agentic monitoring.
Experience with GitHub Enterprise CI/CD, branch strategies, pull request automation, environment promotion, and release management for data pipelines.
Solid Python and/or Scala skills and SQL fluency for Gold-layer transformations and data quality validation.
Hands-on experience designing and orchestrating Azure Data Factory pipelines at scale.
Experience with medallion or lakehouse architecture and disciplined multi-environment deployments.
Preferred experience with Linear, Databricks Mosaic AI, Genie, agentic AI frameworks, MCP server integrations, Cosmos DB, Azure SQL, data governance, lineage, metadata management, legacy BI migration, or real estate and multi-source ERP data.
Strong collaboration skills across engineering, governance, and business stakeholder teams.

Competitive compensation, comprehensive benefits, and flexible work arrangements.
Remote work is available anywhere in the United States except Alaska.
Medical, dental, vision, disability, and life insurance benefits, including low or free basic employee-only medical coverage.
Paid time off including 15 vacation days, 4 personal days, 10 sick days, and 11 paid holidays for new hires.
Birthday off after one year and additional vacation accrued with tenure.
Six-week paid sabbatical after 10 years of service and every five years thereafter.
401(k) with company match up to 6% of pay after six months of service.
Paid parental leave and lifetime fertility benefit reimbursement up to $10,000.
Employee assistance, critical illness, accident, hospital indemnity, pet insurance, legal plans, and charitable giving benefits.

10,000+ employees