crawler

A web crawler used to produce a sitemap containing a list of crawled urls as well as a list of assets each page refers to.

Usage

$ lein uberjar
$ java -jar crawler-0.1.0-standalone.jar [domain]