Engineer II (R-20095)
Dun & Bradstreet1 day ago
Shanghai, ChinaMid Level
Responsibilities
- Design, implement, and maintain Python and Playwright browser-automation collection services.
- Build company update databases, task queues, retry mechanisms, monitoring, and alerting modules.
- Parse, clean, structure, and load collected data according to ODS and enterprise-master-data standards.
- Analyze Ajax, Fetch, XHR, WebSocket, GraphQL, dynamic interfaces, and obfuscated JavaScript to reproduce stable API calls.
- Use Chrome DevTools for breakpoint debugging, request capture, call-stack analysis, and frontend reverse engineering.
- Manage anti-crawling challenges including CAPTCHA, IP blocking, rate limits, webdriver detection, fingerprinting, and request-replay checks.
- Maintain 24/7 collection stability and respond to page, DOM, JavaScript, interface, and field-rule changes.
- Coordinate data validation, field mapping, master-data alignment, and operational reporting with data governance and application teams.
- Evaluate external RPA and third-party data sources and productize reverse-engineering results through configurable templates and failure alerts.
- Participate in occasional releases, evening coverage, emergency support, and related duties.
Requirements
- Bachelor’s degree or above in computer science or a related field.
- At least 3 years of data-collection or web-crawling development experience, including independently building a collection pipeline from zero to production.
- Strong Python skills and familiarity with asynchronous programming, HTTP, HTML, CSS, XPath, and CSS selectors.
- Hands-on experience with Playwright or Puppeteer/Selenium for browser automation and headless deployment.
- Practical anti-crawling experience with CAPTCHA, IP blocking, rate limiting, and Cookie/Session management.
- Experience with proxy IP usage and rotation, task scheduling, queues, retries, monitoring, alerting, logging, and documentation.
- Proficiency with Chrome DevTools, request-flow analysis, interface-signature reverse engineering, and reproducing encryption, signing, or token-generation logic server-side.
- Ability to handle WebSocket, GraphQL, dynamic interfaces, obfuscated JavaScript, and Webpack-bundled code.
- Experience favoring direct interface collection over browser automation when feasible and implementing fallback mechanisms.
Tech Stack
Categories
About Dun & Bradstreet
Dun & Bradstreet builds data and analytics products that help companies assess credit, manage third‑party risk and compliance, and find customers. Its Data Cloud, business credit reports, D‑U‑N‑S Number identity, and APIs are sold as subscriptions and software to finance, procurement, and go‑to‑market teams. Founded in 1841, the company is headquartered in Jacksonville, Florida, and is publicly traded on the NYSE (ticker: DNB).