the-markup/blacklight-collector

Possible addition: Use stealth mode?

johansmith90 opened this issue · 1 comments

Right now the NPM module uses the default puppeteer package. Utilizing puppeteer-extra with the puppeteer-extra-plugin-stealth add-on would allow evasion of certain bot detection functionalities.

Rational: Certain 3rd party tracking and ad scripts alter their behavior if they detect that a visitor is using headless chrome. For example, they do not issue certain network calls or store certain cookies. Utilizing the aforementioned plugin would enable higher fidelity recording of network traffic behavior in those instances.

Happy to submit a PR if it would be of use.

I saw this package recently and it looked good, I didn't have time to test on our infrastructure before launch but its on my list now. Thanks