/crawlarr

Crawlarr is a fast web crawler built in Go. It searches for anchor tags in the HTML pages and follows links. It leverages concurrency to improve speed.

Primary LanguageGoGNU Affero General Public License v3.0AGPL-3.0

Stargazers