7 days ago
Stamford, CT, USAStaff+
Base Salary
$150k - $180k/yr
Responsibilities
- Design and implement advanced web-scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including MFA, CAPTCHAs, and session or token-based login flows.
- Architect scalable scraping pipelines using AWS Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build high-volume data-extraction systems with fault tolerance, retries, and intelligent logging.
- Integrate workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and engineering best practices.
Requirements
- Bachelor’s or master’s degree in computer science, software engineering, or a related field, or equivalent professional experience.
- 10+ years of Python development experience focused on automation and data extraction.
- Expertise with web-scraping tools including Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
- Experience handling multi-step authentication flows, MFA, CAPTCHA solving, and session or cookie management.
- Proficiency deploying and managing workloads in AWS, especially Lambda, S3, IAM, CloudWatch, and Secrets Manager.
- Experience with asynchronous programming, headless browsers, JavaScript-rendered content, web protocols, and reverse-engineering network calls and authentication mechanisms.
- Experience with APIs, JSON/XML, data transformation, CI/CD pipelines, Docker, and infrastructure as code such as CloudFormation and Terraform.
- Familiarity with Apache Airflow or AWS Step Functions for data-pipeline orchestration.
- Experience scraping secure or enterprise-level financial, healthcare, or legal portals.
- Data engineering or ETL experience is a plus.
- Exposure to Python testing frameworks and unit or integration testing.
Benefits
- Base salary range of $150,000 to $180,000, plus equity for full-time employees and an annual performance bonus.
- Employer-matched retirement plan and subsidized healthcare with employer-paid dental and vision coverage.
- Telemedicine, virtual mental health counseling, parental leave, and unlimited paid time off.
- Hybrid schedule with office work Monday through Thursday and remote flexibility on Friday.
