This repository is a template for methodological projects, which require evaluating multiple methods across a range of tasks (defined by problem size, signal-to-noise ratio, etc.), and possibly across a range of metrics (e.g., accuracy and computation time). This repository contains usage examples for many helpful packages and useful practices.
The packages include:
tqdmfor progress bars. See usage inexperiments/result_manager.py.line_profilerfor profiling code. See usage inscratch/profile_mean_vs_std.py.seabornfor prettier plotting. See usage inexperiments/evaluation_manager.py.itertoolsfor simplifying for-loops/list comprehensions.
Some useful practices include:
- A ResultManager which saves/loads the results of running different (potentially time-consuming) algorithms.
- An EvaluationManager which handles plotting results.
- A
setup.shfile which helps make the code easier to reproduce. - Don't use Jupyter notebooks too heavily - they lead to a pretty messy code base. An example
of how I might use it to generate some "final" figures is in
experiments/notebooks/example-notebook.ipynb. - Code/data/result/figure organization.
To see how these are put into use, run:
python3 experiments/result_manager.pypython3 experiments/evaluation_manager.py
The below section shows what I'd usually put into the README.
This repository contains the code for project (link to paper).
To install the necessary requirements in a virtual environment and activate it, run:
bash setup.sh
source venv/bin/activate
The repository is organized as follows:
/algorithmscontains implementations of each algorithm./experimentscontains the code to replicate experiments./scratchcontains files with "scratch work", e.g. profiling and other testing.