monperrus/crawler-user-agents

Why is "Googlebot" not recognized without trailing slash?

egilburg opened this issue · 1 comments

https://support.google.com/webmasters/answer/182072 says Googlebot is used as user agent for the default web crawler.

But https://github.com/monperrus/crawler-user-agents/blob/master/crawler-user-agents.json#L3 only recognizes Googlebot/ with trailing slash.

Thanks.

to differentiate the "main" Googlebot from Googlebot-Mobile, Googlebot-Image, Googlebot-Video, etc.