support broken-link-checker
staabm opened this issue · 2 comments
staabm commented
would be great if this lib would detect https://github.com/stevenvachon/broken-link-checker as a crawler
the user agent is broken-link-checker/0.8.0 Node.js/14.16.0 (OS X; x64)
see https://github.com/stevenvachon/broken-link-checker#useragent
staabm commented
Peferct thx. Couldnt find the ua in the repo :-)