Add missing `CloudFlare` crawlers to bot list
Closed this issue · 0 comments
summercms commented
Below are the following user agents:
CloudFlare-AlwaysOnline/1.0
Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online) AppleWebKit/534.34
Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online)
Cloudflare SpeedTest/1.0
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/79.0.3945.130 Safari/537.36 (compatible; Cloudflare SpeedTest/1.0; +https://blog.cloudflare.com/new-speed-page/)
Cloudflare Diagnostics
Cloudflare Diagnostics
Cloudflare-Diagnostics
Cloudflare-AMP/1.0
Mozilla/5.0 (compatible; Cloudflare-AMP/1.0; +https://amp.cloudflare.com/doc/fetcher.html) AppleWebKit/534.34
Note: Cloudflare SpeedTest uses HeadlessChrome