/Web-Crawler

WebCrawler visits the site with entered URL-address, collects all links from it, visits everyone of them and repeat. Supports multithreading with different threads amount and depth of crawling.

Primary LanguageHTML

Web-Crawler

Hyperskill project

About

The Web is called so for a reason: all these pages are connected to many others through links, resembling a spider’s web. It’s easy to get lost and miss important links, so in this project you will write a program that might help. A Web Crawler collects and saves links from a given page, storing them in the memory for you to access later.

Learning outcomes

You seem like a quite advanced user! Here you will gain even more skills: get to know Swing library and learn how to distribute tasks among the threads and control them.