4 days ago
Remote, Worldwide or Rio de Janeiro, BrazilSenior
Responsibilities
- Scale the CNC data plane into a horizontally scalable fetch scheduler handling geo-routing, site cooldowns, quotas, and 50–100 million requests per day.
- Design and ship the Fleet Manager control plane, including provider adapters, reconciliation, inventory and health management, software rollouts, and BigQuery metrics export.
- Operate and improve the CONNECT proxy gateway, session handling, proxy leases, IP reputation, and request scheduling.
- Engineer mTLS gRPC, long-polling, streaming, NAT and carrier-grade NAT behavior, and bandwidth accounting across unreliable residential links.
- Define Edge SLOs, build peer and bandwidth metrics, and safely run blue/green rollouts of CNC and peers.
- Participate in on-call, lead incident post-mortems, write technical design documents, and review squad designs.
Requirements
- 5+ years of production Rust experience shipping and operating services end to end.
- Experience designing and running schedulers, control planes, or stateful distributed services at scale, with knowledge of consistency, partitions, backpressure, and idempotency.
- Deep knowledge of TCP, TLS, HTTP, gRPC, long-lived connections, NAT, and lossy links at the handshake and packet level.
- Experience building or operating forward proxies, proxy gateways, or residential/ISP proxy infrastructure, including knowledge of IP reputation, session stickiness, and pool health.
- Experience with SLOs, error budgets, observability, production incidents, and on-call operations.
- Comfort running Rust across Windows, Linux, and Android and working with FFI boundaries.
- Strong written communication and problem-solving skills for a globally distributed environment.
- Bonus qualifications include mobile or desktop Rust, fleet or device management, anti-bot and fingerprinting knowledge, web scraping and crawling experience, and open-source contributions.
Benefits
- Remote-first work from anywhere with flexible working hours and high organizational trust.
- Conference attendance and opportunities to meet team members globally.
- Work with a self-motivated, multicultural team and cutting-edge open-source technologies and tools.
Tech Stack
C++Google BigQueryGoogle Cloud PlatformGrafanagRPCJavaKubernetesLinuxmacOSPrometheusPythonRustTerraformWindows
Categories
About Zyte
Zyte builds SaaS and services for extracting structured data from public websites, including Zyte API for crawling/unblocking/extraction, Smart Proxy Manager, and fully managed data projects for enterprise teams. Its business model combines subscriptions with professional services. Founded in 2010 and headquartered in Ballincollig, Cork, the privately held company is the original author and ongoing maintainer of the Scrapy web-scraping framework.