WhichBrowser/Parser-PHP

Add missing regex for search engines and add to separate classes + detect FAKE bots

summercms opened this issue · 0 comments

summercms commented

Real Search Engine Bots

  • Seznam
  • Yandex
  • Sputnik
  • Naver
  • Google
  • Bing
  • Yahoo!
  • Baidu
  • DuckDuckGo
  • Qwantify
  • Coccoc
  • Sogou
  • Yisou
  • Qihoo 360 / 360spider
  • Seekport
  • Apple
  • Istella
  • Soso
  • Vuhuv
  • LetSearch
  • LookSeek / Seeker
  • Yioop
  • Indeed
  • PicSearch
  • TinEye
  • Yacy
  • Gowiki
  • Bytespider
  • Mail.ru

Fake Search Engine Bots

  • Google
  • Yahoo!
  • Yandex
  • Bing
  • Baidu
  • Qwantify
  • Sogou
  • Naver
  • Seznam

Below is example of fake Naver UA:

NaverBot

Real Naver bot uses Yeti see code line:

Parser-PHP/data/applications-bots.php

Line 164 in 880b9fa

[ 'name' => 'Naver Yeti', 'id' => 'yeti', 'regexp' => '/Yeti\/([0-9.]*)/u' ],

Link: https://searchadvisor.naver.com/guide/seo-basic-firewall

Add Fake Naver to Fake Bots PR.

Other Fake Bots

acebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0

facebookscraper/1.0( http://www.facebook.com/sharescraper_help.php)

archive.org_bot

AhrefsBot

LinkedInBot

Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0;+http://www.cloudflare.com/always-online)