/ThingiverseCrawler

A simple script to batch download things from thingiverse.com

Primary LanguagePythonMIT LicenseMIT

Thingiverse Crawler

About:

Thingiverse Crawler is a simple script to batch download things from Thingiverse. It does not use the thingiverse API because non-web application flow is not yet supported.

The script dynamically update wait time to prevent it being blocked by Thingiverse website.

This script has been modified to accomplish the following:

  • Use thingiverse's built in search functionality to grab models matching a user defined query
  • Remove required input of number of models when using the above mentioned search
  • Remove repeated calls to the same page. Original script would loop until it hit the user defined number even if there were no more pages

Example:

Retrieve information of the newest featured 1000 things:

$ ./thingiverse_crawler.py -n 1000

Retrieve information of things matching a search query

$ ./thingiverse_crawler.py -s "kobayashi cube"

Retrieve limited number of things matching a search query

$ ./thingiverse_crawler.py -s "kobayashi cube" -n 1000

A summary.csv will be created. It contians the following information:

  • thing_id
  • file_id
  • file: where to store the output file.
  • license: info about the license it is published in.
  • link: direct link for download.

To download all 1000 files:

$ ./download_model.py summary.csv

To grab context of each thing (i.e. tags, dates, categories):

$ ./grab_context.py summary.csv

To get a single file using its thing_id

$ ./get_file.py thing_id

Modifed By :

Mike Gleason

Web Developer

Original Author:

Qingnan Zhou

New York University

https://www.cs.nyu.edu/~qnzhou/