springload/site-url-scraper

Multithreaded Python application to search a string pattern or find all internal links from a url or CSV list of urls

Python

Issues

Only add pages if not already visited or errored out
#7 opened 8 years ago by loicteixeira
0
De-duplicate links with anchor
#5 opened 8 years ago by loicteixeira
0
Change default file output name and allow overwrite.
#4 opened 8 years ago by loicteixeira
0
Max thread is not respected
#6 opened 8 years ago by loicteixeira
0