A website downloader with a built-in JavaScript engine!
Purpose
Compatibility
Set Up
Functionality
Use Cases
Future Goals
The purpose of Web Loader is to give a user the ability to download portions of or entire websites.
Web Loader is compatible with PhantomJS version 2.1.1 and above.
- Install PhantomJS for your operating system: http://phantomjs.org/download.html.
- Clone this repository.
- You're done! You are now able to run loader.js from the command line.
There are two main ways to download a website with Web Loader:
A host website download involves downloading every page that is present on a given host.
To run a host website download, run a command of the following form:
phantomjs loader.js https://example.com
A path website download involves downloading pages that are in a specific path on the host.
To run a path website download, run a command of the following form:
phatomjs loader.js https://example.com /example/path
There are a few use cases for this software:
Running this program multiple times over a given time period could allow a user to detect changes that are made to a website. This might be useful in tracking when a business announces a new product or updates an application.
The resulting text files generated by this program can be rapidly searched, allowing a user to quickly check if website contains certain verbiage.
In general, this program may be useful in any sort of research that involves extracting verbiage from the internet.
See Projects.