web-scan

Intro

A program that generates a combination of letters from the alphabet and composes an address for accessing the site.After that, it reads the data from the title and description of this site and saves it to the structure. At the end, the program displays all the titles and descriptions of the available sites.

Usage

golang/cmd/go

go get github.com/digital-technology-agency/web-scan

Data store

sqlite - SQLite
jsoneachrow - Json Each row

Generator type

simple - Simple generator type

Init configuration file

$ ./wscan init

Configuration file `config.json`

{
  "process_count": 1,
  "alphabet": "abcdefgefghijklmnop",
  "url_len": 5,
  "concurrency_count": 5,
  "data_store_type": "sqlite",
  "generator_type": "simple",
  "protocol_types": [
    "http",
    "https"
  ]
}

Run

$ ./wscan -configuration_file config.json

digital-technology-agency/web-scan