A generic crawler that attempts to "slurp" the main content and media assets of each page it crawls.
Primary LanguageRuby