/CrawlerMan

Primary LanguageJavaApache License 2.0Apache-2.0

CrawlerMan

A crawler-based search engine.

Requirements

Java 8+
MongoDB

Running the application

gradle (./gradlew) run -PappArgs="['<seed_file>', '<max_URLs_count>', '<number_of_threads>']"

Example

gradle (./gradlew) run -PappArgs="['seed.txt',1000,3]"