21 hours ago
Responsibilities
- Architect, optimize, and own Shopify’s production ingestion platform for operational database and Kafka data entering the analytics data store.
- Use Kafka, BigQuery, GCP, Dataflow, Terraform, and Kubernetes to build high-throughput database and stream ingestion pipelines.
- Partner with database, streaming, governance, infrastructure, and downstream analytics teams to land, validate, and scale data products.
- Optimize cloud spend, source-system load, pipeline complexity, freshness, and completeness.
- Investigate upstream and downstream data-store internals, including indexing, replication, and load patterns.
- Drive technical strategy and team culture, champion projects, represent the team across Shopify, and mentor engineers.
- Participate in the on-call rotation for production ingestion reliability.
Requirements
- Hands-on experience building and operating large-scale data infrastructure with measurable improvements in reliability, throughput, or cost.
- Deep experience with CDC, replication, indexing, partitioning, and operational database behavior under load.
- Strong software engineering skills in Python, Java, and/or other relevant languages, with distributed computing expertise.
- Demonstrated leadership architecting or operating reliable data pipelines at scale.
- Experience delivering production results cross-functionally with database, streaming platform, infrastructure, and downstream data teams.
- Advanced understanding of streaming-system trade-offs among latency, throughput, and cost.
- Experience with Kafka Connect, Debezium, Beam, or Flink is a plus.
- Experience optimizing pipelines across MySQL, Yugabyte, Spanner, or CockroachDB is a plus.
- Experience with BigQuery, Snowflake, or Iceberg and modern warehouse or lakehouse patterns is a plus.
- Experience in e-commerce, large-scale product infrastructure, globally distributed data workloads, or maintaining Scala services is a plus.
- Fluency with AI coding tools such as Claude Code or Cursor is a plus.
Benefits
- Remote-first work arrangement.
- The hiring process aims to complete the interview loop within 30 days, with candidates expected to interview within the week of applying.
- The interview process includes a live pair-programming session, and candidates should bring their own IDE.
Tech Stack
Apache BeamApache FlinkApache KafkaGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPythonScalaSnowflakeTerraform
Categories
BackendData Engineering
About Shopify
Shopify is a leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. Shopify makes commerce better for everyone with a platform and services that are engineered for reliability, while delivering a better shopping experience for consumers everywhere. Shopify powers millions of businesses in more than 175 countries and is trusted by brands such as Allbirds, Gymshark, PepsiCo, Staples, and many more. Find all our jobs here: www.shopify.com/careers