Principal Data Engineer
Black Duck Software, Inc.2 months ago
Belfast, United KingdomStaff+
Responsibilities
- Lead the design and build-out of cross-product data services and a governed customer data plane.
- Build and operate batch, streaming, and event-data ingestion patterns with repeatable source onboarding.
- Own data reliability practices including data contracts, quality checks, lineage, monitoring, and incident response.
- Implement curated datasets, secure query interfaces, product-ready APIs, and versioned customer-facing data products.
- Design and tune data models for operational databases and columnar/OLAP analytics stores.
- Deliver trusted datasets and feature-ready outputs for ML workflows and governed data products for AI automation.
- Partner with product, engineering, and security stakeholders on roadmap priorities and customer outcomes.
- Raise engineering standards through architecture reviews, mentoring, and technical guidance while remaining hands-on.
Requirements
- Significant experience building and operating production data platforms at scale, including on-call and operational ownership.
- Strong SQL and Python skills used to build pipelines, services, and automation.
- Hands-on experience running AWS and Google Cloud systems at the IaaS level, including compute, storage, networking, and IAM.
- Experience with operational databases, analytics stores, performance tuning, data modeling, schema evolution, conformed dimensions, and metric definitions.
- Track record of delivering dependable data products with clear contracts and reliability expectations.
- Experience balancing latency, accuracy, cost, and security in engineering decisions.
- Experience with lakehouse patterns, open table formats, governance, table maintenance, production orchestration, streaming, and safe backfills.
- Familiarity with ML data requirements including training/serving splits, feature-ready datasets, and evaluation datasets.
- Preferred experience with self-service data platforms, catalogs, discoverability, access controls, and regulated or security-sensitive environments.
Benefits
- Hybrid work model in Belfast, UK, with flexible collaboration hours overlapping UK and US time zones.
- Minimal travel.
- Equal opportunity employment and reasonable accommodations for individuals with disabilities.
About Black Duck Software, Inc.
Black Duck Software builds application security tools and services for engineering and security teams, including SAST, SCA, and DAST to find vulnerabilities and licensing/compliance risks in proprietary and open-source code. It offers enterprise software plus on-demand audits and M&A due diligence, delivered in the cloud or on premises. The company is privately held and headquartered in Burlington, Massachusetts.