3 months ago
San Diego, CA, USAStaff+
Base Salary
$125k - $150k/yr
Responsibilities
- Lead modernization of SSIS/SSRS workloads into high-performance cloud-native pipelines.
- Define and implement standardized data design patterns across Lakehouse ingestion, transformation, curation, and consumption layers.
- Develop proofs of concept and points of view for emerging technologies, frameworks, streaming ingestion, and observability capabilities.
- Use Databricks Assistant, Cursor AI, GitHub Copilot, and dbt Mesh AI tests for code generation, testing, documentation, and optimization.
- Integrate predictive models to detect pipeline anomalies and data drift and optimize compute and scheduling.
- Establish CI/CD, version control, peer review, and observability practices across the data platform.
- Collaborate with data architects, platform engineers, analysts, and business product owners to deliver technical solutions.
- Mentor data engineers and promote modern data engineering practices.
- Tune Spark workloads, storage tiers, and orchestration logic across Azure and GCP environments.
Requirements
- Bachelor’s degree.
- 8+ years of experience in data engineering or related technical fields, including at least 3+ years in a lead or senior role.
- Experience designing and implementing CDC, SCD, Medallion, Data Vault, streaming, and batch data patterns.
- Deep expertise with Databricks, Apache Spark, dbt, Fivetran, Census, Airflow, and Kafka.
- Experience with Azure and/or GCP, including technologies such as Synapse, Data Factory, BigQuery, and Pub/Sub.
- Hands-on experience modernizing SSIS/SSRS workloads into cloud-native pipelines.
- Experience building POCs and POVs to validate tools, frameworks, and architectures.
- Working knowledge of AI-assisted engineering tools for development, observability, or optimization.
- Proficiency in SQL and one of Python, Scala, or Java.
- Strong problem-solving, architectural thinking, collaboration, communication, and stakeholder-translation skills.
Benefits
- Benefits may include medical, dental, vision, and life insurance; paid sick leave; approximately three weeks of vacation; about 11 holidays; HSA or FSA accounts; voluntary benefits; a 401(k) plan with employer match; a 529 savings plan; an employee mortgage loan program; and free access to an Axos Bank account with self-directed trading.
Tech Stack
Apache AirflowApache KafkaApache SparkAzureDatabricksdbtGoogle BigQueryGoogle Cloud PlatformJavaPythonScalaSQL
Categories
Data Engineering
