Member of Technical Staff (Backend Software Engineer, Search Crawler)

1 year ago

Berlin, Germany or London, United KingdomSenior

H1B Sponsor

Responsibilities

Own and lead projects involving large-scale web crawlers, ingestion pipelines, and data-processing systems.
Build, maintain, and optimize backend and frontend crawler-service components, including storage, retrieval, and UI dashboards.
Collaborate with Search and Infrastructure teams to ensure reliable, high-quality ingestion and processing of billions of web pages.
Architect and implement full-stack features and scalable distributed systems for high-load and real-time data operations.
Iterate on novel approaches and improve system performance, usability, and reliability.

At least five years of software development experience.
Strong knowledge of data structures and algorithms in Python, C++, Rust, or Go.
Experience with large-scale web crawlers is highly desirable.
Proven experience building, deploying, and optimizing high-load, distributed, and hardware-adjacent services.
Deep understanding of cloud infrastructure with hands-on Kubernetes and AWS experience.
Demonstrated ability to write clean, efficient, and scalable systems.

201-500 employees

The most powerful answer engine. Powering curiosity with answers backed by up-to-date sources. This is where knowledge begins.