A simple workspace to work with Weights & Biases, with automatic CSV dataframe generation.
-
Clone or fork the project
-
Create a new virtual environment (recommended), for example with conda
conda create -n wandb python=3.8 conda activate wandb
-
Set your environment variables
conda env config vars set WANDB_ENTITY=<entity> conda env config vars set WANDB_PROJECT=<project-name> conda env config vars set WANDB_API_KEY=<secret-key> conda activate wandb
-
Install dependencies
pip install -r requirements.txt
The experiments
folder contains one subdirectory per experiment. Feel free to give the experiment directory more explicit names.
experiments/
exp1/
exp2/
exp3/
...
Each experiment subdirectory contains the following:
config.yaml
: the experiment configuration file, defining W&B meta-data (tags, groups, etc.) and fixed parameters of the experiment (e.g. learning rate, number of epochs, etc.)script.py
: script for one jobsubmit.py
: script for submitting the experiment. Use it to launchscript.py
with several seeds, to solve a list of problem instances, etc.README.md
: a brief documentation of the experimentMakefile
In an experiment subdirectory, use make
to automatically launch and parse the experiment. Additionally, you can use the following recipes:
make experiment
to launch the experiment onlymake experiment NOTES=<message-note>
to launch the experiment with a note in W&B dashboardmake parse
to parse the current experimentmake clean
to runwandb gc
make cleanall
to clean and remove all artifacts of the current experiment
From the project root or from the experiments
folder, use make experiments
, make parse
, make clean
or make cleanall
to launch the corresponding recipe on all experiments.
The script submit.py
dumps the id of the submitted jobs in a YAML file jobs.yaml
. In an experiment subdirectory, make parse
creates a dataframes/
folder and export the dataframes corresponding to the jobs in jobs.yaml
(the latest runs for this experiment)
metrics.csv
contains the history of all logged valuessummary.csv
contains the logged values for the last time stepsystem_events.csv
contains the logged system events.