about 2 months ago
São Paulo, BrazilMid Level / Senior
Responsibilities
- Develop, test, and deploy robust web scraping scripts and crawlers using advanced Python tools.
- Architect and maintain asynchronous scraping systems for large-scale data extraction.
- Implement and optimize anti-blocking strategies and proxy rotation for reliability.
- Manage and automate data ingestion pipelines and integrations with external REST APIs.
- Debug, monitor, and improve scraper performance and data quality.
- Collaborate with engineers to enhance scraping infrastructure and monitoring systems.
- Assist with DevOps tasks, including Docker, CI/CD, and managing Linux environments.
Requirements
- Proven experience in high-volume web scraping and data extraction using Python.
- Solid understanding of HTML parsing, browser automation, and asynchronous programming.
- Proficiency with web scraping frameworks like Playwright, Scrapy, or Selenium.
- Strong knowledge of REST APIs, HTTP protocols, and proxy management.
- Familiarity with SQL and NoSQL databases for data storage and processing.
- Experience with Docker, Linux environments, and version control (Git).
- Fluent in English, both written and spoken.
- Self-driven, detail-oriented, and capable of owning significant projects.
Benefits
- High degree of freedom and opportunity for meaningful impact in a growing business.
- Flexibility with remote work support.
- Competitive compensation package and support for personal and professional development.
- Great work atmosphere within a small, talented, and international team.
- Optional modern office located near Berlin, easily accessible by public transport.