Crawler Application is a simple crawler application (Console Application) allows to Crawl a paths of a given website using symfony framework and Spatie\Crawler Library.
-
First of all Download this console application and make it inside HTDOCS folder and run the XAMPP server.
-
the limited crawling number is 20 , and you can change this number from setMaximumCrawlCount() method inside CrawlerCommand.php class located on src/Crawler/Commands folder.
-
you should enter a valid website link for example : "https://www.linkedin.com"
-
open the command line and go to CrawlerApp folder, after that run the symfony server using this command :
php bin/console server:run -
tap this command line to start the application : php bin/crawler start
-
after that it will shows you this message "Please enter a valid website link with 'https://' :" , so you should enter a valid website link with "https://" , and tap ENTER button.
-
and finally you can see the result in the command line in addition to that you can see it also inside a JSON file located on the main folder of CrawlerApp application named "crawler json result.json".