GrepJob
Tavily

Software Engineer - Web Crawling

Tavily
Apply
2 months ago
Tel Aviv-Yafo, IsraelMid Level / Senior

Responsibilities

  • Build distributed data acquisition systems that capture and structure the live web.
  • Investigate and analyze browser internals and anti-automation systems.
  • Prototype and deploy intelligent automation frameworks using Playwright and Puppeteer.
  • Conduct research into network flows and JavaScript-based protections.
  • Collaborate with AI and infrastructure teams to integrate real-time web data.
  • Translate technical insights into production-grade components.
  • Evolve Tavily's capabilities in web interaction resilience and speed.

Requirements

  • 3–5 years of experience as a backend or systems engineer with large-scale infrastructure.
  • Strong programming skills in Python or Node.js; Go or C++ is a plus.
  • Experience building and maintaining browser automation systems in production.
  • Solid understanding of browser internals and network protocols.
  • Experience designing high-performance, resilient systems for scale and concurrency.
  • Strong debugging and analytical skills for dynamic web conditions.
  • Excellent documentation and communication skills.

Tech Stack

C++DockerGoKubernetesNode.jsPython

Categories

BackendData EngineeringDevOps