crawler-detection

There are 12 repositories under crawler-detection topic.

hisorange/browser-detect
Browser Detection for Laravel by hisorange!
Language:PHP1.1k 26 145146
scrapy-plugins/scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
Language:Python366 55 4190
loadkpi/crawler_detect
Ruby gem to detect bots and crawlers via the user agent
Language:Ruby140 2 814
x-way/crawlerdetect
Golang module to detect bots and crawlers via the user agent
Language:Go65 3 36
stephenafamo/isbot
Detect bots/crawlers/spiders using the user agent string.
Language:Go19 3 01
BryanMorgan/isbot
Rust library to detect bots using a user-agent string
Language:Rust16 2 01
zivdar001matin/web-crawler-detection
Final project of Rahnema College internship. Crawler detection using unsupervised learning methods.
Language:Python15 1 00
loonies/corrode
A web crawler bot killer
Language:Python2 2 00
S4k1dl0/Bot-Analytics-with-PHP
A PHP-based application for detecting and logging bot activity on your website. This project utilizes CrawlerDetect to differentiate between human users and crawlers, storing relevant data in a MySQL database.
Language:PHP1 1 00
manuelxaguilar/bot-inspector
Simple node user-agent inspector that detects bot requests
Language:JavaScript0 2 00
RominaShin/Machine-Learning-Internship
A 2 months internship program at rahnema college
Language:Jupyter Notebook0 1 00
vlados/laravel-blade-crawler-detect
Simple package for adding directives to show/hide content from crawlers
Language:PHP0 2 00