/goodbots

Verify IP addresses of respectful crawlers like Googlebot by reverse dns and forward dns lookups

Primary LanguageGoApache License 2.0Apache-2.0

Stargazers