2 days ago
Responsibilities
- Architect, optimize, and own Shopify’s production ingestion platform for operational database and Kafka data.
- Build and operate high-throughput, continuously reliable database and streaming ingestion pipelines.
- Use Kafka, BigQuery, GCP, Dataflow, Terraform, and Kubernetes to improve platform performance and efficiency.
- Partner with database, streaming, governance, infrastructure, and downstream analytics teams to deliver scalable data products.
- Optimize cloud costs, source-system load, pipeline complexity, freshness, and completeness.
- Investigate upstream and downstream data-store internals, including indexing, replication, and load behavior.
- Drive technical strategy, champion projects, represent the team across Shopify, mentor engineers, and collaborate with senior data platform engineers.
- Participate in the on-call rotation for production ingestion reliability.
Requirements
- Hands-on expertise building and operating large-scale data infrastructure with measurable reliability, throughput, or cost improvements.
- Deep experience with CDC, replication, indexing, partitioning, and operational database behavior under load.
- Strong software engineering skills in Python, Java, or other relevant languages, with a distributed systems mindset.
- Demonstrated leadership architecting or operating reliable data pipelines at scale.
- Experience delivering cross-functional production results with database, streaming, infrastructure, and data consumer teams.
- Advanced understanding of streaming-system trade-offs involving latency, throughput, and cost.
- Preferred experience with Kafka Connect, Debezium, Beam, Flink, MySQL, Yugabyte, Spanner, CockroachDB, BigQuery, Snowflake, Iceberg, Scala services, e-commerce or globally distributed workloads, and AI coding tools such as Claude Code or Cursor.
Benefits
- Remote-first work arrangement.
- Fast hiring process with interviews expected within one week and an intended completion of the interview loop within 30 days.
- Live pair-programming interview using the candidate’s own IDE.
- Participation in an on-call rotation is required.
Tech Stack
Apache BeamApache FlinkApache KafkaGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPythonScalaSnowflakeTerraform
Categories
Data Engineering
About Shopify
Shopify is a leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. Shopify makes commerce better for everyone with a platform and services that are engineered for reliability, while delivering a better shopping experience for consumers everywhere. Shopify powers millions of businesses in more than 175 countries and is trusted by brands such as Allbirds, Gymshark, PepsiCo, Staples, and many more. Find all our jobs here: www.shopify.com/careers