fcavallarin/htcap

Make possible to resume a crawl with an existing database

GuilloOme opened this issue · 3 comments

The actual behavior is:

  • if no .db file create a new one.
  • if there is an existing one, create a new one with rename.
  • if option -w, overwrite the existing one.

Benefit

  • start and stop a crawl without re-crawl the whole site
  • having a single db for multiple assessment

It's already possible to have multiple assessment on a single database. Every call to "htcap.py scan ..." will generate a new assessment on the same db. What is not possible is to resume the crawling using an existent database.
This issue should be a feature request asking for functionality to resume an interrupted crawling.

Cool, I mainly focus on the crawl feature, so I didn't notice that it was possible in scan mode.

I updated the issue name to "resume a crawl" and I started to work on a pull-request (since we need it in our workflow)

Thanks,

I have a branch waiting to be turned in a Pull-Request, but since it depends on #17, I'll wait before summiting it.