xpresso/ruby-crawler

Collect urls within a domain to generate sitemaps. Google sitemaps for example.

Ruby