JayBizzle/Crawler-Detect

Agent with '+http...' almost always a bot.

bartonlp opened this issue · 1 comments

Any time you find a user agent with '+http' it is almost always a bot. So for example, '+https://help.something.com' will probably be a bot. The 'good' bots almost always have a place where you can see who they are.

"Probably" is a bit loose for my liking, unless you can convince me otherwise 🤔