A console app to download a local copy of a website.
The user provides the URL. The program downloads the URL to a local copy, and tries to analyze it for further links to follow on the same domain. Rinse and repeat till all links are exhausted.
The files are stored under local-copies folder created where the app runs.
The app also creates a report JSON file that holds some metadata, errors that happened while scanning, and a list of all URLs that was scanned.
The report is stored in the same folder as the executable, and is called scan-report-{yyyy}-{MM}-{dd}-{HH}-{mm}-{ss}.json.
You either run it through the dotnet CLI, by invoking it as such:
dotnet run <url>
Example:
dotnet run https://microsoft.com
Or, of course, compile your own executable and just run it with the URL as the parameter.
There are two options apart from the URL.
Set timeout for all HTTP calls. Defaults to 5 seconds.
-t <MILLISECONDS>
Set path to store the local copies and reports. Defaults to current working directory.
-p <PATH>