5 days ago
Menlo Park, CA, USASenior
Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using advanced SQL and Python.
- Build, optimize, and maintain data workflows across Hive/Spark-based data warehouse environments.
- Develop and manage Dataswarm, Airflow, or equivalent pipeline orchestration workflows, including scheduling, backfills, and failure troubleshooting.
- Create and maintain analytics-ready datasets and data models for workforce, staffing, capacity, and productivity reporting.
- Monitor pipeline health, implement data quality checks, and establish monitoring practices aligned with data accuracy and SLA requirements.
- Maintain legacy pipelines, automate manual workflows, and build self-service datasets.
- Partner with data scientists, data architects, analysts, and operations stakeholders to translate requirements into reliable data solutions.
- Participate in internal practice meetings, thought leadership, client case studies, learning materials, networking events, and Digital Transformation practice development.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, or a related technical field is required.
- Strong proficiency in advanced SQL, including Presto, Trino, or Spark SQL.
- Basic to intermediate Python programming experience for production ETL/ELT development.
- Experience building, optimizing, and maintaining large-scale data pipelines in Hive/Spark-based data warehouse environments.
- Understanding of data warehousing fundamentals, partitioning, incremental loads, data modeling, and query performance optimization.
- Experience with pipeline orchestration tools such as Dataswarm, Airflow, or equivalent platforms.
- Experience troubleshooting pipeline failures, managing historical backfills, implementing data quality checks, and maintaining reliable workflows.
- Understanding of data pipeline monitoring, reliability, and SLA requirements.
- Preferred experience includes Unidash, Tableau, Git, data quality frameworks, anomaly detection, SLA alerting, and self-service datasets.
- Preferred experience includes operations and support analytics and collaboration with data scientists, analysts, and business stakeholders.
- Familiarity with Six Sigma, CBPP, BPM, ISO 20000, ITIL, or CMMI certifications is preferred.
Benefits
- Six-month duration.
- Onsite work in Menlo Park, California.
- Direct W2 candidates only.
- Comprehensive medical benefits, dental and vision coverage, life insurance, a 401(k) plan with matching, paid holidays, networking, and career learning and development programs.
Tech Stack
Categories
Data Engineering
About itD Tech
itD Tech is the technology organization within the Idaho Transportation Department, building and operating digital services and enterprise systems for Idaho’s transportation network. It develops and maintains tools for DMV transactions, permitting, GIS/asset management, traffic and safety data, and statewide infrastructure operations, funded by state and federal transportation budgets. Headquartered in Boise, it supports internal agency staff, motorists, carriers, contractors, and local partners across Idaho.