15 hours ago
Responsibilities
- Architect, optimize, and own Shopify’s production ingestion platform for operational database and Kafka data.
- Build and operate high-throughput, continuously reliable database and stream ingestion pipelines at large scale.
- Use Kafka, BigQuery, GCP, Dataflow, Terraform, and Kubernetes to improve platform performance and efficiency.
- Partner with database, streaming, governance, infrastructure, and downstream analytics teams to land, validate, and scale data products.
- Drive cost optimization and reduce cloud spend, source-system load, and pipeline complexity without sacrificing freshness or completeness.
- Investigate upstream and downstream data-store internals, including indexing, replication, and load patterns.
- Lead technical strategy, champion projects, mentor engineers, and collaborate with senior Data Platform engineers.
- Participate in the on-call rotation for production ingestion systems.
Requirements
- Proven hands-on expertise building and operating large-scale data infrastructure with measurable reliability, throughput, or cost improvements.
- Deep experience with data stores, including change data capture, replication, indexing, partitioning, and operational database behavior under load.
- Strong software engineering skills in Python, Java, or other relevant languages, with a systems and distributed-computing mindset.
- Demonstrated leadership architecting or operating reliable data pipelines at scale.
- A track record of cross-functional impact with database, streaming platform, infrastructure, and downstream data-consumer teams.
- Advanced understanding of streaming-system trade-offs involving latency, throughput, and cost.
- Preferred experience with Kafka Connect, Debezium, Beam, Flink, MySQL, Yugabyte, Spanner, CockroachDB, BigQuery, Snowflake, Iceberg, Scala services, e-commerce or globally distributed workloads, and AI coding tools such as Claude Code or Cursor.
- Applicants should be prepared to interview within the week, complete the interview loop within 30 days, and participate in a live pair-programming session using their own IDE.
Benefits
- Remote-first work arrangement.
- Participation in an on-call rotation supporting production systems.
- Hiring process target of completing the interview loop within 30 days.
Tech Stack
Apache BeamApache FlinkApache KafkaGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPythonScalaSnowflakeTerraform
Categories
Data Engineering
About Shopify
Shopify is a leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. Shopify makes commerce better for everyone with a platform and services that are engineered for reliability, while delivering a better shopping experience for consumers everywhere. Shopify powers millions of businesses in more than 175 countries and is trusted by brands such as Allbirds, Gymshark, PepsiCo, Staples, and many more. Find all our jobs here: www.shopify.com/careers