/mercator-crawler

A Mercator crawler with url-frontier, url-set, meta-data fetcher, etc. inspired by the Stanford Information Retrieval book

Primary LanguageTypeScriptMIT LicenseMIT

Watchers