Sr Data Engineer
Baker Tilly15 days ago
Portland, OR, USA +7 moreSenior
Base Salary
$185k - $230k/yr
Responsibilities
- Develop and maintain scalable batch and incremental pipelines for structured and unstructured enterprise data.
- Build and optimize lakehouse processing, transformation layers, data models, and integrations with enterprise and legacy sources.
- Implement automated data validation, reconciliation, lineage tracking, metadata management, data contracts, and provenance controls.
- Develop identity-resolution processes and scalable models for client, engagement, and document data.
- Implement data cataloging, classification, permissions, and access controls for sensitive enterprise information.
- Build knowledge graphs, search indexes, vector or hybrid retrieval pipelines, and related data relationships for AI applications.
- Create pipelines for AI agent activity, practitioner feedback, corrections, ground-truth datasets, and evaluation workflows.
- Establish testing frameworks, benchmarks, quality standards, and validation processes for extraction, retrieval, and grounding accuracy.
- Implement incremental processing, change data capture, versioning, monitoring, and data freshness controls.
- Deploy, monitor, troubleshoot, and maintain production pipelines using automated testing, code reviews, and operational documentation.
- Collaborate with AI engineering, platform and production engineering, enterprise data teams, subject matter experts, external partners, and globally distributed engineering teams.
- Transition data solutions into internally owned, documented, reusable, and maintainable production environments.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field; an advanced degree is preferred.
- At least 3 years of hands-on data engineering or software engineering experience.
- Demonstrated experience independently building, deploying, operating, and maintaining production pipelines on a modern lakehouse platform.
- Experience implementing data quality, lineage, data contracts, automated testing, reconciliation, validation, or traceability in production pipelines.
- Strong proficiency in Python and SQL.
- Strong understanding of scalable and maintainable data modeling principles.
- Experience designing and developing batch and incremental pipelines for structured and unstructured enterprise data.
- Strong software engineering fundamentals, including version control, automated testing, code reviews, debugging, and reliable repeatable workflows.
- Ability to validate technical solutions through tests, benchmarks, experiments, performance metrics, or comparative evaluations.
- Preferred experience with Databricks, Apache Spark, Delta Lake, medallion architecture, and Microsoft Azure data platforms.
- Preferred familiarity with data catalogs, classification, access controls, lineage, Unity Catalog, or comparable governance solutions.
- Preferred experience with knowledge graphs, graph databases, graph data modeling, vector search, hybrid search, retrieval applications, document extraction, content chunking, or search indexing.
- Preferred experience with entity resolution, master data management, change data capture, or incremental data processing.
- Understanding of data provenance, traceability, and governance requirements in complex enterprise or regulated environments.
- Strong analytical thinking, technical judgment, troubleshooting, communication, documentation, and cross-functional collaboration skills.
- Ability to manage competing priorities, adapt to changing requirements, and deliver in a fast-paced collaborative environment.
- Experience with AI-assisted or agentic development tools, or willingness to incorporate them into daily development.
- Prior AI, machine learning, LLM, or knowledge graph experience is not required; no professional certifications or licenses are required.
Benefits
- Baker Tilly offers a comprehensive compensation and benefits package to eligible employees.
- The posting directs candidates to discuss work location status and requirements with a Baker Tilly talent acquisition professional.
Tech Stack
Categories
Data Engineering
About Baker Tilly
Baker Tilly provides audit/assurance, tax, and advisory services—including analytics, governance risk and compliance, and government contracting—to companies, public sector entities, and nonprofits. Headquartered in Chicago and organized as a partnership, it operates coast to coast in the U.S. and is an independent member of Baker Tilly International, with U.S. attest services delivered by Baker Tilly US, LLP and tax and advisory by Baker Tilly Advisory Group, LP.