Detect and verify select (i.e. Google
and Yandex
) search bots. Similar functionality is provided by the
browser
gem; this goes further in that we use DNS lookups to validate that a bot
is actually what it says it is.
Googlebot
. Detected and verified as suggested here.Yandex robot
. Detected and verified as suggested here.
Simply add search-bot-detector
to your Gemfile
:
gem "search-bot-detector"
Super simple example:
require "search-bot-detector"
# IP address of `example.com` returns false.
SearchBotDetector.new("93.184.216.34", "Googlebot").any_bot?
# => false
# IP address of `crawl-66-249-66-1.googlebot.com` returns true.
SearchBotDetector.new("66.249.66.1", "Googlebot").any_bot?
# => true
# IP address of `spider-100-43-81-144.yandex.com` returns true.
SearchBotDetector.new("100.43.81.144", "+http://yandex.com/bots").any_bot?
# => true