/web-crawler

Embark on an adventure into the depths of the web with this powerful web crawler made with Node, Axios and Cheerio.

Primary LanguageJavaScriptMIT LicenseMIT

web-crawler

A Basic Web-Crawler Attempted in Node.js (Libraries used axios and cheerio). The Project demonstrates core web crawling functionalities:

  • fetching
  • parsing
  • link extraction
  • iterative link following
  • crawl limit control
  • success-error ratio calculation.

(working in progress)