Grader

This project has been retired! Find out more.

Simple setup (using Xcode and no docker/worker agents -- frontend only)

Install vapor and dependencies

If you are on macOS then make sure you have Homebrew installed first. Then:

brew tap vapor/homebrew-tap
brew update
brew install vapor
brew install sqlite3

Clone

Do a git clone of this repo.

Setup Xcode project

To create the files need

vapor xcode

Open the xcodeproj file in Xcode, check that "Run -> My Mac" is selected as the target and then hit "Play".

Check it is running in your browser at http:://localhost:8080.

Setup database

The database will be created automatically when you run the project.

If you want to run the project in Xcode, then you can use a SQLite database. The base Config/fluent.json is already setup to use the sqlite driver by default. Docker will use the driver defined in Config/docker/fluent.json, which is set to MySQL by default.

To seed the tables:

vapor run seed

Adding dependencies

If you change Package.swift then use vapor update to download dependencies.

Serious setup

You will need Docker (at least version 17).

This setup uses docker-compose to create 4 containers:

database - a standard mariadb image
redis - a standard redis image
web - our vapor image that serves the website
worker - our vapor image that runs multiple job threads

Orchestrate your containers

Clone the repo
Open your terminal at the repo root, and cd docker
Build the base docker image docker build -t apptitude/vapor vapor
Build the application ./build
Start redis & database docker-compose up -d redis database
Check they're up docker-compose ps
Check the logs if you have problems docker-compose logs -ft <container_id>
After database is up, start web docker-compose up -d web
After web is up, start worker docker-compose up -d worker
Open your browser at http://localhost or your docker VM IP address
By default you will be using mysql and the db will be empty, so next you should seed the db docker-compose run worker run seed --env=docker

Workflow

Every change to the Swift source code requires compilation and restart the web/worker containers. The script ./build will stop, compile and start.
Changes to the resources (html, css, js, images -- and Leaf!) do not require recompilation (just refresh the browser).
To inspect the database, it is easiest to login directly to the database container docker-compose exec database mysql -u root -p grader
To test the running of submission jobs (or manually run a job), login to the worker container docker-compose exec worker bash and then use the vapor command vapor run submission X where X is your submission id.

Other commands

Compile our custom Vapor image (required before we start using docker run apptitude/vapor):

docker build -t apptitude/vapor vapor

docker run -it --volume=$PWD/..:/app --entrypoint /bin/bash  apptitude/vapor

Run Vapor commands (e.g. vapor build):

docker run -it --volume=$PWD/..:/app apptitude/vapor build

In general, you can run any Vapor command (e.g. vapor XXX):

docker run -it --volume=$PWD/..:/app apptitude/vapor XXX --env=docker

Run a one-off worker:

docker run -it --volume=$PWD/..:/app apptitude/vapor run worker --env=docker

Deployment

This was tested on AWS, to a Ubuntu Xenial 16.04 (20170414) (ami-8fcc75ec) instance. The security group should have http and ssh open.

ssh into your newly created instance. (Note: the user to login with for Ubuntu images is ubuntu.)
Install docker

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y --allow-unauthenticated docker-ce
sudo systemctl status docker
sudo usermod -aG docker ${USER}

and check it is working docker

Install docker-compose

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo curl -o /usr/local/bin/docker-compose -L "https://github.com/docker/compose/releases/download/1.15.0/docker-compose-$(uname -s)-$(uname -m)"
sudo chmod +x /usr/local/bin/docker-compose

and check it is working docker-compose -v

Create the /app directory

sudo mkdir /app
sudo chown ubuntu:ubuntu /app

Sync the files (e.g. copy from your local machine to the instance). See ./deploy for an example.
ssh back into the instance, and build/start the docker services

cd /app/docker
docker build -t apptitude/vapor vapor
docker run -it --volume=$PWD/..:/app apptitude/vapor build
docker-compose up -d

Check everything looks ok on the logs.

docker-compose logs -ft

Updates to the server

Make a backup first

ssh grader
cd /app/docker
docker-compose exec database bash
mysqldump -u root -p grader > dump.sql
[Ctrl-D]

Copy backup down to localhost

docker cp `docker-compose ps -q database`:/dump.sql /app/dump.sql
[Ctrl-D]
scp grader:/app/dump.sql ./

Make database changes (from aws) -- if required

ssh grader
cd /app/docker
docker-compose exec database mysql -u root -p grader
[Ctrl-D]

(and do you stuff!)

Resync files (logout to localhost first)

./deploy

Rebuild the worker container (if required)

ssh grader
cd /app/docker
docker-compose up -d --no-deps --build worker
[Ctrl-D]

Resource limits

Use these commands on running containers to modify the resource allocation.

docker update --cpus=".25" <containerid>   # Guarantee that a container has at least 25% of 1 cpu core
docker update -c 512 <containerid>         # Give 50% share of the CPU to a container (512 out of 1024)
docker update -m 750M <containerid>        # Limit memory to 750MB for a container

antonyharfield/grader