1 month ago
Base Salary
$159k - $302k/yr
Responsibilities
- Own end-to-end architecture and implementation for the Security Data Platform on Databricks and AWS.
- Build and operate distributed batch and streaming pipelines for EDR telemetry, network logs, identity events, cloud audit logs, and other security data sources.
- Write and review production Python and SQL, develop reusable engineering patterns, and lead code and design reviews.
- Build and maintain tested Terraform modules for AWS and Databricks infrastructure, including S3, IAM, VPC networking, and MWAA.
- Design lakehouse data models, normalization patterns, data contracts, quality checks, cataloging, classification, lineage, access controls, and retention policies.
- Operate Airflow and MWAA orchestration, including dependencies, retries, backfills, alerting, automated recovery, and service-level objectives.
- Diagnose complex production issues across Spark, storage, orchestration, and cloud infrastructure, and improve incident response and recovery readiness.
- Lead cost, performance, reliability, capacity, and latency improvements through workload profiling, query and Spark tuning, lifecycle policies, partitioning, and compute right-sizing.
- Partner with security and infrastructure teams to deliver data interfaces and pipeline capabilities for SIEM modernization and automated detection deployment.
- Mentor engineers, lead cross-team technical discussions, resolve tradeoffs, and contribute to technical hiring assessments.
- Evaluate platform innovations including AI-assisted engineering and operational automation while protecting sensitive security data.
Requirements
- Typically 10 or more years of experience building and operating production data or software engineering platforms, with demonstrated Staff-level scope and impact; equivalent depth and outcomes are welcome.
- Demonstrated experience personally delivering complex distributed systems and leading full-system design and implementation across team boundaries.
- Deep production expertise with Spark, Databricks, Delta Lake, lakehouse architecture, Python, and advanced SQL.
- Strong AWS experience across storage, compute, networking, and IAM, plus production experience designing, testing, and maintaining reusable Terraform modules.
- Practical knowledge of distributed systems concepts including partitioning, replication, consistency, back pressure, scalability, and fault tolerance.
- Experience with Airflow, MWAA, or equivalent orchestration, as well as observability, data quality, automated testing, deployment, and recovery practices.
- Demonstrated ability to improve platform cost, performance, and reliability with measurable results while balancing security and consumer requirements.
- Strong technical communication, cross-functional influence, design review leadership, mentoring, disagreement resolution, and adoption-building skills.
- Preferred: experience with security telemetry, OCSF or comparable security schemas, SIEM, SOAR, threat detection, or investigations.
- Preferred: experience with Unity Catalog governance, data classification and retention, regional data controls, or multi-region platform design.
- Preferred: experience with Cribl Stream, Vector, or similar telemetry routing and enrichment tools.
Benefits
- U.S. base salary range is $159,200-$301,600 annually, with location-dependent pay; California range is $208,300-$301,600.
- Non-sales roles may be eligible for the Annual Incentive Plan and certain roles may receive a new-hire equity award.
- Adobe provides comprehensive benefits programs and emphasizes accessibility, inclusion, and employee impact.
Tech Stack
Categories
Data Engineering
About Adobe
Adobe builds software for creative work, document workflows, and customer experience used by individuals, teams, and enterprises. Its products include Creative Cloud (Photoshop, Illustrator, Premiere Pro), Acrobat/Document Cloud, and Experience Cloud for marketing and analytics, sold primarily via subscriptions. Founded in 1982 and headquartered in San Jose, California, Adobe is a public company traded on NASDAQ under the ticker ADBE.