/canvas-scraper-cli

A NodeJS command-line tool for scraping and downloading data from Canvas (e.g. assignments and modules).

Primary LanguageJavaScript

Canvas Scraper CLI

A NodeJS command-line interface for scraping and downloading data (e.g. assignments and modules) from a Canvas course.

Dependencies

Canvas-Scraper uses Puppeteer, a headless browser, to navigate and scrape data from Canvas. This requires some form of Chromium to be available on the system. The easiest way to do this is by installing Google Chrome.

Getting Started

You'll first need to get the cookies for your current Canvas session to allow the scraper to have credentials to your Canvas. This needs to be done in JSON format (an example can be found in cookies-example.json).

The easiest way to do this is by logging into Canvas in your browser and using an extension to export your current cookies (e.g. CookieManager for Chrome).

Using Pre-Compiled Release

First, download the latest version from the Releases page and extract the ZIP file. Inside the folder, you'll find three (3) different executables, with each being for a different operating system.

To use the executable, you can either open it using your file explorer (which will open a wizard to guide you through the different arguments and flags) or simply navigate to the directory in a shell and call the executable.

e.g Windows
./canvas-scraper.exe [options] <url>

Using Source Code

You will need NodeJS and NPM.

Once installed, download the project dependencies using...

npm i

After, you can simply run the entry file directly...

node index.js [options] <url>

Usage

Usage: canvas-scraper [options] <url>

Scrape data from a canvas course

Arguments:
  url                      Course Homepage URL (e.g. https://<school_domain>/courses/<course_id>)

Options:
  -o, --output <dir_name>  output directory name (default: "courses/course")
  -c, --cookies <path>     path to cookies file (default: "cookies.json")
  -a                       scrape assignments (default: false)
  -m                       scrape modules (default: false)
  -h, --help               display help for command

You must use the 'a' and/or 'm' flags, otherwise no data will be scraped.

Download File Structure

  • ASSIGNMENTS
    • [<Assignment> (<(Grade %) or (Points Earned/Total Points) or (N/A) or (✅ or ❌)>)]
      • .ASSIGNMENT
        • .ASSIGNMENT.pdf
        • {Embedded Files}
      • .COMMENTS.txt
      • .SUBMISSIONDETAILS.pdf
      • {Submission Files}
    • .ASSIGNMENTS.pdf
  • MODULES
    • MODULE SECTION
      • MODULE NAME
        • .MODULE.pdf
        • {Embedded Files}
    • .MODULES.pdf
  • .HOMEPAGE.pdf

Video Tutorial

Watch the video below for a quick guide on how to download and use the CLI!

Video Tutorial Thumbnail