You'll need docker
and unzip
to use this repo. For docker
, visit the official guide to get started. For unzip
, you can install it on ubuntu via sudo apt-get install unzip
.
Install the python package by
git clone https://github.com/ProKil/anlp-grading-tools
cd anlp-grading-tools
pip install -e .
To evaluate your code, you'll need to change the environment variables in test.sh.
ANLP_TMP_DIR
: mkdir
a new folder, e.g. mkdir tmp
, and point this variable to the absolute path of the tmp folder.
SUBMISSION_DIR
: this should point to the folder containing your submission zip file. Note that the toolkit will automatically evaluate all zip files in the folder.
SCORES_DIR
: this should point to an empty folder. Your score will be logged in a text file there.
DATA_DIR
: this should point to the data folder of minbert-assignment
.
Example code to prepare the folders:
mkdir tmp
mkdir scores
cp -r path/to/minbert-assignment/data ./
mkdir submission
cp your/submission.zip submission
Now you can evaluate your code through bash test.sh
, after which your scores are at SCORES_DIR/andrewid
.
- You may find writing files inside
ANLP_TMP_DIR
andSCORE_DIR
requiring permission. You can either usesudo
or log into docker throughdocker run -v FOLDER_TO_WRITE:/mnt -it --entrypoint /bin/bash anlp
andcd /mnt
to write those files. - You may experience other permission issues with docker. Please refer to this page to use docker without sudo.