/web-crawler

Barebone implementation of web-crawler using Breadth First Search Algorithm

Primary LanguageJava

web-crawler

To understand network topology of any given site

  • This is based on BFS algorithm
  • Vertex are replaced by website-name
  • Run App.java to see how it works

What are we trying to solve through this application?

  1. What are the most frequently visited websites for a given website?

Note

  • If the application fails to print out any URLs, check the input URL that is fed in the first place
  • Please note that some servers deny access to read from URL class method
  • So an exception is thrown occasionally, but nevertheless the application will not stop working