This repository includes generic functions for interacting with the Canvas LMS as well as instructions for a highly individualized and specific grading setup. Where possible I have tried to distinguish between the two but there are still places where I have not adequately separated them.
Also, this package is fairly unstable due to the low quality of the code :-/. Expect API’s to continue to change.
This package is developed with emacs 27.0.50 and org-mode from git; there’s no guarantee it will work with other versions, sorry :-(.
It also depends on request.el and org-mime, both of which can be downloaded from MELPA. Request.el in particular has changed a lot in the last little while (mid-2019) so the MELPA version is required.
As of <2019-09-30 Mon> dash, ts, and let-alist are also all dependencies.
I personally use password-store to manage my oauth token and userid, and I recommend that method for others as well.
Once you have the dependencies, then simply
- clone this repository
- add the directory to your load-path
- require org-lms
This is my use-package declaration; you’ll have to adjust the path of course:
(use-package org-lms
:load-path "/home/matt/src/org-grading"
:pin manual
:after (org password-store ts dash let-alist)
:commands (org-lms-setup org-lms-get-courseid)
:config
;; nothing yet
)Canvas has a rich RESTful API, which accepts and returns JSON representations of many kinds of course-related objects, including students, assignments, announcements, and pages. A basic function, org-lms-canvas-request, uses Emacs’s built-in JSON.el and the powerful request.el library to standardize the formats of requests and responses. Then, for each supported API endpoint, org-lms has a set of convenience functions:
- one or more getters for retrieving data from Canvas, usually named
org-lms-get-* - one or more setters for posting or modifying data to Canvas, usually named
org-lms-post-* - one or more transformers for modidying the data provided by canvas into a form that’s useful for local use. Some of these may become less necessary as I adapt my local schema to come into closer alignment with Canvas, but there will always be extra information that Canvas cannot easily store (Github ID’s, nicknames, etc.).
At present, this package provides only a very limited set of functions related to these categories:
- courses
- students
- assignments
- announcements
In order to use them, you’ll first need to set the following variables:
org-lms-baseurl, a system-wide variablethat represents your institution’s Canvas API URLorg-lms-token, your secret API token, which you can acces svia the Canvas web interface (follow the link for details)ORG_LMS_COURSEID, set either as a file-wide keyword or a headline-level org property. You can useorg-lms-get-coursesto retrieve all the course objects and their:idproperties.- other ID’s for individual students, assignments, submissions, announcements, etc; the higher-level functions here will retrieve those values and store them in headline-level properties.
Canvas also has a GraphQL API which is dramatically more efficient than the RESTful API. I would like to ocnvert som portions of this oce to use that API, but that will take a while. See the Github Issue in this repo for more details.
Create a course-specific directory containing your org-mode source files. In each file, add the #+ORG_LMS_COURSEID: value specific to your course. This will have to be set each year, since the course ids will change.
Write your announcements and assignments as org-mode subtrees, and then post them to canvas with the appropriate convenience functions. Hopefully the most important information from the API response will then be recorded in the headlines.
See ./Grading-template.org for some examples.
Follow the instructions above, navigate to your syllabus document, and simply M-x org-lms-post-syllabus. If everything’s working properly, the syllabus should appear on the front page of your Canvas site.
This function assumes he syllabus lives in its own file.
I’ve added a method to export subtrees as HTML announcements. This saves me having to compose in the web interface (finally!). Just call org-lms-headline-to-announcement from the right subtree (sorry, it won’t traverse up the tree like ox-hugo's “dwim” scope – haven’t programmed that yet!). A successful post will set the ORG_LMS_ANNOUNCEMENT_ID property, and the next time you call from this subtree, the existing announcement will be updated (that is, no new announcement will be posted). Also, the ORLG_LMS_ANNOUNCEMENT_URL property will be set, and a new browser tab will open up with the announcement page.
Pages are pretty easy to create from subtrees; just execute org-lms-post-page from a subtree and its content will be uploaded as HTML to your Canvas site.
Marking is the bulk of the work associated with an LMS and the most complex part of the workflow this package is designed for. These instructions may not be up to date and may also be somewhat idiosyncratic.
See Grading-template.org and Assignments.org for an example of how to set up assignments. The functions defined here expect each assignment to be a subtree. They will look for a number of headline properties and file-wide keyword values before making the API call:
In order to use this system, assignments need to be created as org-mode headlines and posted to the LMS via =org-lms=. Otherwise the metadata that org-lms relies on for retrieving student work won’t be present. If you have another workflow, you might want to modify some of the existing functions.
Assuming you keep all your assignments in a file Assignments.org in the root directory of your course repo, do the following:
- add a keyword line
#+ORG_LMS_COURSEID: XXXXsomewhere in your org file (I prefer the top). You can easily get the course ID just by inspecting the URL of your course, or by using theorg-lms-WHATISITAGAINfunction - In each assignment headline, you’ll want to set a number of properties. This is somewhat tedious, so I recommend creating a template and modifying later:
:DUE_AT: 2018-11-23 :GRADING_TYPE: letter_grade :OL_PUBLISH: t :ASSIGNMENT_TYPE: canvas :ASSIGNMENT_WEIGHT: 0.10 :CANVAS_SUBMISSION_TYPES: (online_upload) :PUBLISH: t :GRADING_STANDARD_ID: 458
Let’s go through these one by one. They are a little repetitive and should probably be rationalized.
- DUE_AT sets a due date. Right now, the time component is hard-coded to be 11:59PM EST on that date. This should be fixed!
- GRADING_TYPE is required ify ou want to use letter grades.
- If using letter grades, then GRSADING_STANDARD may also be necessary
- OL_PUBLISH is nil by default, though maybe that should be changed
- CANVAS_SUBMISSION_TYPES is a list object and must be set if you intend to ocllect student work via Canvas.
- ASSIGNMENT_TYPE should be set to `canvas` unless you are collecting work some other way
- Canvas also requires that an ASSIGNMENT_WEIGHT be set, or it won’t record marks properly.
- OL_DIRECTORY is the directory in which to collect student work. It defaults to a downcased, whitespace-free transformation of the assignment name (that is, the headline content) and will later be created in the main Grading directory if it doesn’t exist (see below).
Once you’ve set the metadata, go ahead and write the assignment. If you include a subheading tagged `rubric` then that subheading will be used later by org-lms when constructing grading headlines (see the next section).
When you’re done, post your work to Canvas with org-lms-parse-assignment (misleading name, should be changed!), and, importantly, parse the assignments file with =org-lms-save-assignment-map=. This will create an emacs-lisp file whose sole contents are an alist containing a representation of the assignment
I generally use a file called Comments.org and keep it in a directory Grading which I exclude from the main git repo for my course (obvious reasons). This file also needs to have certain metadata set:
- courseid with
#+ORG_LMS_COURSEID: NUMBER - location of assignments org file with
+#ORG_LMS_ASSIGNMENTS: PATH(Note: this is a *change from earlier practice, andorg-lms-setupwill no longer work if you do this!) - the variable
org-lms-merged-studentsshould also be set. This is a little baroque and should be streamlined; the name derives from my perhaps idiosyncratic practice of maintaining a student list that includes both nicknames and github ids for students. Right now, the easiest way to generate this list involves creating a filestudents.csvand running(org-lms-merge-student-lists)to sync the existing csv and the current list of students (which will change every time someone adds or drops the class). - run
org-lms-setup-gradingto generate a table of assignments. You can then manually click the “create headlines” field
- [X] Write assignment(s) in
Assignments.org(see template) - [X] generate an
assignments.elfile from the WIM contents ofAssignments.org, and ideally - [X] automaticlaly write to this file every time I upload an assignment
- [X] inside the grading template,
- [X] having set the location of
assignments.elas a file-level keyword variable, - [X] read its contents and
- [X] use them to generate headlines.
- [X] having set the location of
So, this is roughly finished. Now just need to add a few more keywords to make everything run smmmoooooottthh as butter.
Each local assignment has as its cdr a plist which will be used to construct the grading document & to handle a variety of grading-related tasks. Here is the initial structure of an assignment:
(test .
(:name "Test Assignment"
:directory "response-paper-1"
:weight 0.10
:grade-type "letter"
:submission-type "canvas"
:rubric-list ("Organization" "Clarity of Argument"
"Grammar and Spelling" "Grade"
"See Attached Paper for further Comments")
))
- :name
- used both to construct the headline for the assignment, and to associate the local assignment with a Canvas assignment object
- :directory
- local storage of student work
- :weight
- used in constructing final grades (not implemented)
- :grade-type
- one of “letter”, “number”, or “passfail” – but not yet implemented properly
- :submission-type
- one of “email” “github”, or “canvas”. Should be used in the future for handling (a) attachment of student files and (b) return of student works. Right now there’s no canvas implementation.
- :rubric-list
- This is what I started with – My grading rubrics are all definition lists, with comments entered at the end of the list entry.
Run org-lms-merge-assignments to add a few extra properties from an associated Canvas assignment. I’m not yet able to automate the creation of these assignments, though that should be possible.
There is now preliminary support for uploading assignments to a course. This is very much a work in progress.
I have two existing systems for marking student work:
- students email me their papers or submit via Dropbox. I collect the papers in a single directory.
- Students submit work via Github Classroom. I bulk-clone their repos and mark via PR comments
It would be nice to replace the first of these with a system for downloading papers directly from Canvas. I’m working on that right now. This is now implemented! Use org-lms-get-canvas-attachments to getthese. Now I need to hook it up to org-lms-make-headinges.
Running (org-lms-make-headings assignment-name) will generate org heading trees with the following structure:
- Assignment Name
- Student Name 1
- Student Name 2
- etc
Each headline will have a number of properties set to make marking easier. Existing student papers will be attached to the grading subtree and can be quickly opened with C-c C-a o. I find the workflow very quick and easy. I have libreoffice configured with a few shortcuts for commonly used editing markup (checkmarks, smileyfaces, paragraph marks, and expansion shortcuts for “wrong word” and “awkward”). PDFs are much slower for me to mark, as neither pdf-view nor evince has really excellent text annotation UI. TThis may be a limitation of the PDF annotation standards. For github repos, the PR interface is quite rich for code; for text work it’s a little bit clumsier, but I don’t have a solution for that yet.
The org agenda is a powerful tool for sorting and filtering headlines. Since each student assignment is a headline, we can mis-use the agenda to rapidly navigate to specific places in a comments file. So, for instance, to find all trees with a grade of “0” enter the following:
C-c a < m GRADE="0"
This will open the agenda, searching only the current buffer for matches, matching the property “GRADE” with value “0”
It should be possible to add this to org-agenda-custom-commands, but I’m having trouble with this right now.
Also having trouble matching with “|” (“or”).
Once created, the agenda can be filtered, e.g., search for a partiular student using = and then entering a substring of the student’s name
Unfortunately I’m not very adept right now with the agenda but maybe this wil lget easier!!
Right now I run org-lms-mail-all to mail out all subtrees marked with a READY org-todo state. This is generally fairly reliable, though sometimes there are issues with the message queue.
NEW: I have written org-lms-put-single-submission-from-headline which half-works and is ready for testing.
The canvas API does not accept all HTML5 semantically-named entities.
(let ((entities (json-read-file "/home/matt/entities.json")))
entities)I’m experimenting with using literate programming and org-babel-tangle as my main development modality. It’s convenient to navigate the various parts of the code, etc. But it may be hard to submit PR’s etc.
The Canvas API is described in the offocial docs, which lives in a different form here. Here is a tutorial on using Postman to test canvas lms, and an official Getting Started guide. Here’s a similar resouce organized as a course. Examples in these docs mostly use cURL. Instead we are using request.el (github) and restclient (github) (where appropriate) for inspecting api requests. helpful stackexchange intor to restclient. some advanced org-mode restclient shit to aspire to.
I have had a hell of a time parsing api results; important to always set json-key-type, json-object-type, and json-array-type before invoking json-read. In request this has to be done in the parser declaration which is a pain. Otherwise lists end up as vectors, which sucks b/c I don’t knw how to use vectors in lisp!
Anyway, some progress being made in sample code in ./Grading-template.org, but still have abunch of progress to make on this front!!
In case I forget, you gneerate oauth tkens in the settings pane on quercus.
There’s still some shitty junk in here
THis is abig one – not sure how to do this!!
also makes it easier to give the directory w/ student papers & the headline different names
this would e nice!
a little bit difficult
generating assignments is too finicky right now. I’d like to be able to do it form the Assignments page (!!)
WOuld be nice if the marks were a bit easier to read in email clients
and move it to the right place. But this in each repo or maybe just run from repo.
Shouldn’t be so har.d Also add all comments PRs if you can. Hig