/crawler4jUI

A basic webcrawler based on crawler4j with easy use GUI

Primary LanguageJava

Crawler UI

  • Crawler UI is a java swing based UI for a basic html crawler that uses popular crawler4j API. Simple UI

Run the code

  • Download or clone the repository
  • Create a Maven project in eclipse and import the project files
  • Run GUIDesign.java

Usage

The application takes 2 required and 4 optional parameters.

  • Seed URL(required)
  • Data storage location(required)
  • Number of crawling threads
  • Maximum depth of crawling
  • Maximum number of pages to fetch
  • Politeness delay(in ms)