18 days ago
Remote, CanadaStaff+
Responsibilities
- Own the package identity data model, including coordinate schemas, cross-ecosystem mapping, deduplication, and re-discovery.
- Architect and evolve streaming and batch pipelines, including migrations, infrastructure choices, cost optimization, and reliability improvements.
- Design and maintain versioned, backward-compatible data contracts for downstream teams.
- Lead ingestion of commercial vendor SBOM feeds and collaborate with external engineering teams on specifications.
- Evaluate and apply AI/ML techniques to identity re-discovery, package deduplication, and data-quality problems where appropriate.
- Use AI-assisted engineering tools for coding, code review, investigation, and data exploration.
- Lead reliability practices including alarm triage, SLO discipline, cost management, and incident response.
- Set technical direction for a small team, own systems across multiple release cycles, and influence peer teams without direct authority.
Requirements
- 10+ years of professional software engineering experience, including 3+ years at Staff level or with equivalent technical leadership scope.
- Deep experience building and operating data platforms at scale, including streaming and batch pipelines and distributed storage.
- Strong Java proficiency or a systems-programming background that enables rapid productivity.
- Experience designing cross-team data contracts, schemas, versioning, backward compatibility, and upstream data coordination.
- AWS-native operating experience, including EMR, S3, SNS, and SQS, with strong cost and performance judgment.
- Comfort with ambiguity, vendor-facing technical work, and collaboration with external engineering teams.
- Working fluency with Claude or equivalent AI-assisted engineering tools and sound judgment about using LLMs versus classical ML.
- Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
- Experience working in an agile environment and collaborating cross-functionally.
- Preferred experience with Maven internals, npm, PyPI, Go modules, RPM/Debian, CycloneDX, VEX, SPDX, entity resolution, deduplication, record linkage, or relevant open-source contributions.
Benefits
- Parental leave policy.
- Paid volunteer time off.
- Diversity and inclusion working groups.
- Flexible working practices.
Tech Stack
Categories
Data Engineering
