Le. C.V., Pardos, Z.A., Meyer, S.D., Thorp, R. (in-press) Communication at Scale in a MOOC Using Predictive Engagement Analytics. To appear in Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED). London, UK. pdf
- Clone the repo and run
npm install
- Fill out scripts/startup.sh
- Match frontend/client_main.js and frontend/cross_main.js with the info from the startup file
- Upload
client_main.js
,cross_main.js
,crossfilter.js
,d3.js
, andstyles.css
to edX
- Click Files & Uploads which is under the Content dropdown at the top of the page in the vertical studio editor
- Insert a Raw HTML Block
- Copy and paste
client.html
into the block - Change scripts/weekly.sh paths to the proper locations and then run to get the user roster
- Run process/make_user_info.py which uses the roster to get the user_info.csv file and places it in the data directory
- Change scripts/daily.sh paths to the proper locations and then run to get yesterday's event log
- Run process/make_prediction.py which uses the event log to get the predictions.csv file and places it in the data directory
The startup script will automatically pull down from edX the roster on a weekly basis and create user_info.csv file. It will also pull down from edX the event log on a daily basis and create a predictions.csv file, which will be anonymized.
user_info
path andpredictions
path can be changed in scripts/startup.shsecretUsername
andsecretPassword
from server.js need to be copied to frontend/cross_main.js. This is to ensure that only you can receive the predictions info, which just in case is also anonymized.- Set the
emailUsername
,emailPassword
, andsmtp
if you choose not to masquerade and use a mail service like MailGun. - Change
pkey
,pcert
,cert
with the correct paths to enable HTTPS (required for edX) - Set the port for the server and make sure it matches with frontend/client_main.js
- Start the mongo database on a different screen
mongod --dbpath=db_data --port=1301
- Launch with
node server.js
RNN_event_list.csv
: A list with all the relevant event typesall_events.log
: A continually concatenated list that has all user events in the course. Created by scripts/daily.sh which is run by scripts/startup.shuser.sql
: A sql file of all the users in the system for the most recent week. Created by scripts/weekly.sh which is run by scripts/startup.shuser_info.csv
: File of all the up to date user info. Created by process/make_user_info.pyprediction.csv
: File of the predictions of completion, attrition, and certification for anonymized users. Created by process/make_prediction.py
- Upload
client_main.js
,cross_main.js
,crossfilter.js
,d3.js
, andstyles.css
to edX
- Click Files & Uploads which is under the Content dropdown at the top of the page in the vertical studio editor
- Insert a Raw HTML Block
- Copy and paste
client.html
into the block
The React frontend is a work in progress intended to be integrated into edX. To build, do the following:
cd
tofrontend_react
- Run
npm install
to install dependencies - Copy the contents of
.env.example
to a file called.env
, and fill out the relevant credentials inside - Run
npm run start
to start the dev server - Run
npm run build
to build the production, minified build. The resulting build will be output to thebuild
directory withinfrontend_react
and can be served by any static web server like Nginx.
Communicator.jsx
contains the vast majority of form code that takes the selected filters and interacts with the server to request/save/send emails to students.Charts.jsx
contains most of thed3
code necessary to render the bar charts, selection bars, and filtering functions.
Filtering is done by crossfilter2
, making it fast and scalable (< 5ms for 1k records on a Core i5)
Architecture and weights for the three trained models used to predict attrition, certification, and completion.
- Must run
python make_user_info.py
beforepython make_prediction.py
prediction_utils
: Various utility functions that allowmake_prediction.py
to work
automated.py
: Checks if any new students meet policies that has been designated with "automatic check"daily.sh
: Grabs the event log file from AWSweekly.sh
: Grabs the user file from AWSstartup.sh
: Does the following:- Sets the path for prediction and user_info files
- Runs
daily.sh
+automated.py
+make_prediction.py
to updatepredictions.csv
daily - Runs
weekly.sh
andmake_user_info.py
to updateuser_info.csv
weekly