/web-crawler

A simple web crawler to crawl the internet

Primary LanguagePHP

Simple Web crawler

  • run below line on cmd php index.php
  • you will see the below line Init url:
  • write/input your URL to start crawling from example: Init url: https://en.wikipedia.org

To store files in a file(pages.json),

write below command on cmd: php.exe index.php > pages.json