/web-crawler

fun task for learning CompletableFuture functionality

Primary LanguageJava

Web Crawler

Test task for fun

The​ ​ command-line​ ​ program​ ​ should​ ​ do​ ​ the​ ​ following:

  • Read a ​string (search term) from standard input
  • Get a Google result page for the search term
  • Extract main result links from the page
  • Download the respective pages and extract the names of Javascript libraries used in them
  • Print top 5 most used libraries to standard output

How to use

./gradlew run

and then type your google query for the program

or

./gradlew run --args=<your google query here>