/aiocrawler

A simple async crawler

Primary LanguagePython

A simple async link scanner

how to use

  • cd into repo, setup venv and activate
  • pip install -e .
  • run aiocrawler https://example.com to crawl or aiocrawler --help to see usage