/WebGenome

A breadth first web crawler that stores HTTP headers in a MongoDB database with a web front end all written in Go.

Primary LanguageGoGNU General Public License v2.0GPL-2.0

Stargazers