🌱 goClone

goClone is a powerful utility that enables you to effortlessly download entire websites from the Internet and save them to your local directory. With goClone, you can easily obtain HTML, CSS, JavaScript, images, and other associated files directly from the server and store them on your computer.

One of the standout features of goClone is its ability to accurately preserve the original website's relative link structure. This means that when you open any page of the "mirrored" website in your browser, you can seamlessly navigate through the site by following links just as if you were browsing it online.

goClone empowers you to have offline access to websites, making it convenient for various purposes such as research, archiving, or simply enjoying a website without an internet connection.

So go ahead, give goClone a try and experience the freedom of having your favorite websites at your fingertips, even when you're offline!

MacOS installing

$ brew install shurco/tap/goclone

Alternately, you can configure the tap and install the package separately:

$ brew tap shurco/tap
$ brew install goclone

Manual

# go get :)
go get github.com/shurco/goClone
# change to project directory using your GOPATH
cd $GOPATH/src/github.com/shurco/goClone/cmd
# build and install application
go install

Examples

# goclone <url>
goclone https://domain.com

Usage

Usage:
  goclone <url> [flags]

Flags:
  -b, --browser_endpoint string   chrome headless browser WS endpoint
  -c, --cookie                    if set true, cookies won't send
  -h, --help                      help for goclone
  -o, --open                      automatically open project in default browser
  -p, --proxy_string string       proxy connection string
  -r, --robots                    disable robots.txt checks
  -s, --serve                     serve the generated files using gofiber
  -P, --servePort int             serve port number (default 8088)
  -u, --user_agent string         custom User-Agent (default "goclone")
  -v, --version                   version for goclone

Making JS Rendered Requests

JS Rendered requests can be made using -b flag. For example start image :

docker run -d -p 9222:9222 --rm --name headless-shell chromedp/headless-shell

then run goclone:

goclone -b "ws://localhost:9222" https://domain.com