linkedtales/scrapedin

Good reason for removing random agent

rileyai-dev opened this issue · 2 comments

I can see that browser agents have been commented out...

Is there a good reason why you don't randomize agents?

The reason can think of, is that if your user agent keeps changing that it might be easier to detect by LinkedIn. So keeping it fixed to what your account typically uses could make more sense that rotating through a list of (most likely) outdated and over-used user agents..
But also interested to know why it's been commented out :)

The reason can think of, is that if your user agent keeps changing that it might be easier to detect by LinkedIn.

Yes, that was my intention, but honestly, it was just a guess 😅 , that's why it's commented. I got that change agent idea from another scraper so it may be effective after all, honestly, I don't have any idea about it!

Do you guys think it's better to randomize again?