/goodreads

gocolly/colly demo project, crawl quotes from https://www.goodreads.com/quotes

Primary LanguageGoMIT LicenseMIT

goodreads crawler

Go Report Card Coverage codeql-analysis LICENSE Godoc

About The Project

usage:

  • proxy-server: optional
  • tag: optional, if empty, all tags will be visited
  • debug: optional, enable debug mode?
> go build .
> ./goodreads -h
a simple goodread quote crawler

Usage:
  goodreads [flags]

Examples:
./goodreads --proxy-server http://127.0.0.1:1081 --tag hope --debug

Flags:
      --debug                 debug mode?
  -h, --help                  help for goodreads
      --proxy-server string   proxy server to use
      --tag string            optional: specify a tag to scrape?

> ./goodreads --proxy-server http://127.0.0.1:1081 --tag hope --debug

then check the goodreads.csv file