set the depth of website to visit
Closed this issue · 2 comments
I just have a question.
I checked two parameters(limit_spidering, max_relative_links_to_visit) in the etc/honeyclient.xml.
But, I think that these two parameters can't control the depth of website to visit.
I want to set the depth of website to visit, and the depth is vaild for the urls listed in the url.txt.
In other words, I don't want to recount the depth for the intermediate links.
I wonder whether there are some parameters or not for this?
If not, what shoud I do?
Thank you.
Author: kindlund
Hi song,
Built-in crawling support has been phased out of the Honeyclient package. You're probably using an older version, which still had residual code from our beta crawler.
Instead, we recommend you use a external crawler, such as (http://crawler.archive.org/), which you could feed the Honeyclient as a list of URLs.
Regards,
-- Darien
Author: anonymous
Hi Dairen, thank you for your fast response.
I'll try to use the external crawler recommended by you.
Regards,
-Song