googleweblight not detected as bot
JPMulder opened this issue · 1 comments
Hi I've gone through this list
https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers
Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko; googleweblight) Chrome/38.0.1025.166 Mobile Safari/535.19
Was not detected as bot
weblight is not a bot (in the traditional sense).
See here - https://developers.google.com/search/docs/advanced/mobile/web-light#robots
Key info from that page...
The Web Light user agent is used only for explicit browse requests of a human visitor, and so it can ignore robots.txt rules, which are used to block automated crawling requests.
So it is a "bot", but it only visits when triggered by a human request, in which case, you wouldn't want to flag it as a bot.