4 months ago
Responsibilities
- Architect, optimize, and own Shopify’s production ingestion platform for operational database and Kafka data.
- Use Kafka, BigQuery, GCP, Dataflow, Terraform, and Kubernetes to build high-throughput database and stream ingestion pipelines.
- Partner with database, streaming, governance, infrastructure, and downstream analytics teams to land, validate, and scale data products.
- Optimize cloud costs, source-system load, pipeline complexity, freshness, and completeness.
- Investigate upstream and downstream data-store internals, including indexing, replication, and load patterns.
- Drive technical strategy and culture, champion projects, represent the team across Shopify, and mentor engineers.
- Participate in the on-call rotation for production ingestion systems.
Requirements
- Proven hands-on expertise building and operating large-scale data infrastructure with measurable improvements in reliability, throughput, or cost.
- Deep experience with CDC, replication, indexing, partitioning, and operational database behavior under load.
- Strong software engineering skills in Python, Java, and/or other relevant languages, with a distributed systems mindset.
- Demonstrated leadership architecting or operating reliable data pipelines at scale.
- Track record of cross-functional impact with database, streaming platform, infrastructure, and downstream data teams.
- Advanced understanding of streaming-system trade-offs among latency, throughput, and cost.
- Preferred experience with Kafka Connect, Debezium, Beam, Flink, MySQL, Yugabyte, Spanner, CockroachDB, BigQuery, Snowflake, Iceberg, Scala services, and globally distributed data workloads.
- Preferred fluency with AI coding tools such as Claude Code and Cursor.
Benefits
- Remote-first work arrangement.
- Opportunity to work on Shopify’s AI-native data platform and infrastructure used across the company.
- Interview process targeted for completion within 30 days, with candidates expected to interview within the week of applying.
- Live pair-programming interview requiring candidates to bring their own IDE.
Tech Stack
Apache BeamApache FlinkApache KafkaGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPythonScalaSnowflakeTerraform
Categories
Data Engineering
About Shopify
Shopify is a leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. Shopify makes commerce better for everyone with a platform and services that are engineered for reliability, while delivering a better shopping experience for consumers everywhere. Shopify powers millions of businesses in more than 175 countries and is trusted by brands such as Allbirds, Gymshark, PepsiCo, Staples, and many more. Find all our jobs here: www.shopify.com/careers