VaclavRut/sitemap-sniffer
Crawler that will check for most common URLs where are sitemaps, which simplifies the crawling when you know the sitemap.
JavaScript
No issues in this repository yet.
Crawler that will check for most common URLs where are sitemaps, which simplifies the crawling when you know the sitemap.
JavaScript
No issues in this repository yet.