Arena

This repository contains the tools necessary to run MegaMinerAI Tournaments.

GCP Project

Before running the Arena, you must have a Google Cloud Platform project set up. This should already be done, but if not, here are the steps:

Create new project
Enable APIs and Services on dashboard (scroll down, left side)
Click +Enable APIs and Services at top
Scroll down and click Compute Engine API
Click ENABLE and wait
Click Create Credentials at the top right
Compute Engine API, Yes Manually create the firewall rules, so terraform doesn't destroy them.

Go to navigation menu, VPC network → Firewall rules

New Rule

name: allow-webservices

network: default

target tags: allow-webservices

allow

source filters: ip ranges: 0.0.0.0/0

protocols:

tcp: 80, 443, 8080, 8000

icmp

Now, there is a default Compute Engine API service account.

Create a key for it as a JSON. Save this in the secrets folder.

Go to navigation menu, IAM & Admin → Service Account → actions → create new key → save as JSON
Create main.tf, replacing the project name with your own.
The resource google_compute_instance default should work.

Ansible Piece:

Generate SSH key: ssh-keygen -t rsa -b 4096 -C "ansible"
apt install xsel
cat ~/.ssh/id_rsa.pub | xsel -ib
Go to Compute Engine -> Metadata -> SSH keys -> Add new one and paste

Assuming you have pip installed... 5. pip install ansible 6.

INSTALL jq or not actually... Continued... I added the output variable to terraform. Now I am seeing how to osLogin I gave the service account the osloginadmin role https://cloud.google.com/compute/docs/instances/connecting-advanced#thirdpartytools Following that tutorial... Get the external IP

Okay so I believe that OSLogin will not work for us - it says you need an org account which idk if we wanna rely on. I have disabled OS Login and know that to make ssh keys work, you need to give them to the instances SO now main.tf does that, and we use our ansible ssh key to connect However, you also need to provide the public key to Compute Engine -> Metadata -> SSH Keys You SSH by doing ssh ansible@IP -i secrets/ansible ansible config has host key checking off because these vms are getting made and destroyed and so dont pass checks all the time.

PYRENA:

Pyrena is a small arena for siggame/cadre written in Python

REQUIREMENTS:

Before using this arena, you must do the following:

Setup a postgres server and create the tables defined in https://github.com/siggame/ophelia/blob/develop/db/init.sql
Setup a siggame gameserver https://github.com/siggame/Cerveau
- Ideally this would run on the same machine as the pyrena so that all network calls are local, though this is not a requirement
Install the psycopg2 python3 module pip install psycopg2
Install docker on the machine running Pyrena

COMPONENTS:

Tournament_scheduler : Schedules games for N-elimination tournaments in the BD Webserver backend : Serves API used by client web app PostgreSQL DB : Stores state for users, teams, games, etc. Pyrena : Reads scheduled games from database, sets up docker VMs and runs them against each other using Gamerserver, uploads logs to droopyz Gameserver : Listens for game clients, validates commands, updates clients, generates gamelogs Client web app : Authenticated user-friendly interface for uploading submissions, viewing gamelogs, etc. Droopygz : Stores and serves large gzip-compressed files

PROVISIONING:

General procedure to setup a fresh pyrena VM:

Create VM with
- ubuntu 1804
- recommended 25GB boot disk (for gameserver's gamelogs)
Allow access to database from this new VM if using IP-based DB whitelisting
SSH into machine and install packages:

sudo apt-get update sudo apt-get install tmux sudo apt-get install python3-psycopg2 git clone https://github.com/tarnasa/pyrena git clone https://github.com/siggame/joueur sudo apt-get install docker.io sudo groupadd docker sudo usermod -a -G docker $USER

Setup the gameserver

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
*new shell*
nvm install 9.11
npm init
npm install
npm run-script build
vim node_modules/@types/express-serve-static-core/index.d.ts
line 900, replace fn: BLABLABLA with fn: Function
echo '#!/bin/sh' > run_gameserver
echo npm run start -- --arena --visualizer-url=http://vis.codegames.ai --only-load blobmaster >> run_gameserver
chmod +x run_gameserver

Start the gameserver

./run_gameserver

Create a run file for pyrena, something similar to: You will need to fill in the DB_HOST. This will be the database IP that will have the competitors names in it.

#!/bin/sh
export LOOKBACK_SECONDS=60
export GAME_NAME='Type name here'
export MATCH_TIMEOUT=280
export DOCKERFILE_PATH=/home/coolcat/joueur
export DB_NAME=postgres
export DB_USER=ophelia
export DB_PASS=postgres
export DB_HOST='Ask web team. This will be an IP'
export DROOPY_URL=http://drop.megaminerai.games/
export LOGFILE_PATH=/home/coolcat/pyrena_logs
export SUBMISSION_CACHE_PATH=/home/coolcat/submission_cache
export DROOPY_CREDS=
export RUN_FOREVER=True
export CONTAINER_CPU=1
export CONTAINER_RAM=1g
python3 pyrena.py

Test the pyrena by running it with RUN_FOREVER=False Pyrena will:
Read from database a list of teams and submissions
Write a new row into GAMES and GAMES_SUBMISSIONS table
Download zip files for clients
Unzip
Verify the directory structure
Replace Dockerfile in the submission using language-specific file in DOCKERFILE_PATH
Build the docker container for each client
Upload build log to droopy and DB with path to file
Setup a session in the gameserver
Start the docker containers and connect them to the gameserver
Poll gameserver till match is over or timeout
Upload gamelog, client logs to droopy
Update GAMES table with fail or success and link to gamelog
Quit

The pyrena provides lots of logging, so any errors should be easily apparent.

Run pyrena continuously

Using the run-file above Pyrena will repeat the above steps until until a fatal error, such as the disk filling up.

Run a tournament

Mark all teams you want in the tournament as eligible. The tournament schedule will use the latest submitted version for each team.

GAME_NAME=Chess \
DB_HOST=\
DB_PORT=\
DB_NAME=\
DB_USER=\
DB_PASS=\
N_ELIMINATION=3\
BEST_OF=7\
python3 tournament_scheduler.py

siggame/Arena

Arena

GCP Project