Link to the documentation: https://spine-toolbox.readthedocs.io/en/latest/?badge=latest
Spine Toolbox is an open source Python package to manage data, scenarios and workflows for modelling and simulation. You can have your local workflow, but work as a team through version control and SQL databases.
Spine Toolbox is released under the GNU Lesser General Public License (LGPL) license. All accompanying documentation, original graphics and other material are released under the Creative Commons BY-SA 4.0 license. Licenses of all packages used by Spine Toolbox are listed in the Spine Toolbox User Guide.
If you use Spine Toolbox in a published work, please cite the following publication (Chicago/Turabian Style).
Kiviluoma Juha, Pallonetto Fabiano, Marin Manuel, Savolainen Pekka T., Soininen Antti, Vennström Per, Rinne Erkka, Huang Jiangyi, Kouveliotis-Lysikatos Iasonas, Ihlemann Maren, Delarue Erik, O’Dwyer Ciara, O’Donnel Terence, Amelin Mikael, Söder Lennart, and Dillon Joseph. 2022. "Spine Toolbox: A flexible open-source workflow management system with scenario and data management" SoftwareX, Vol. 17, 100967, https://doi.org/10.1016/j.softx.2021.100967.
We provide three options for installing Spine Toolbox. The first two options also require that you follow the Pre-installation steps:
- Python/pipx (we intend to make stable releases every month or so)
- From source files (this is the cutting edge - and more likely to have bugs)
- Windows executable as .zip bundle (requires only unzipping the downloaded .zip file)
- Windows installation package (these are quite old - not recommended)
These steps apply to both Python/pipx option and to From source files option.
-
If you don't have Python installed, please install it e.g. from Python.org.
-
Test that python is now in your PATH. Open a new terminal (e.g. Command Prompt) window and type
python --version
The output should be the Python version. If it did not work (e.g. the output is 'python' command not found or similar or the Microsoft Store opens), you need to add the Python installation folder into the PATH environment variable. On Windows, this can be done from Windows Control Panel (use search with 'environment'). See specific instructions for all operating systems from this link.
-
Install Git. Windows version can be found here: https://git-scm.com/download/win.
-
Test that Git is now in your PATH. Open a new terminal after installing Git and type
git --version
If the output from this command is not the Git version, you need to add
<git-install-path>/Git/cmd
into your PATH environment variable. -
Upgrade
pip
to the latest version (pip is a package manager for Python)python -m pip install --upgrade pip
This works best for users that just want to use Spine Toolbox and keep it updated with new releases.
Once you have completed the Pre-installation steps, do the following:
-
Install pipx. pipx helps in creating an isolated environment for Spine Toolbox to avoid package conflicts. Open a terminal and run
python -m pip install --user pipx
After pipx has been installed, run
python -m pipx ensurepath
-
Restart the terminal or re-login for the changes of the latest command to take effect.
-
Choose which Spine Toolbox version to install. Latest release version from PyPi is installed using
python -m pipx install spinetoolbox
or the latest development version using
python -m pipx install git+https://github.com/spine-tools/spinetoolbox-dev
That’s it! To launch Spine Toolbox, open a terminal and run
spinetoolbox
If for some reason the command is not found, the executable can be found under ~/.local/bin
(%USERPROFILE%\.local\bin
on Windows).
To upgrade Spine Toolbox to the latest release, open a terminal and run
python -m pipx upgrade spinetoolbox
or upgrade the development version with
python -m pipx upgrade spinetoolbox-dev
This option is for developers and other contributors who want to debug or edit Spine Toolbox source code. Once you have completed the Pre-installation steps, do the following:
-
Clone this repository. Open a terminal, use the
cd
command to change the current directory to some data directory (preferably not a OneDrive folder on Windows), and rungit clone https://github.com/spine-tools/Spine-Toolbox.git
This creates a
Spine-Toolbox
directory into your current directory. -
Use the
cd
command to go toSpine-Toolbox
dircd ./Spine-Toolbox
-
Create a Python virtual environment using
python -m venv .venv
This creates a
.venv
directory into theSpine-Toolbox
directory, but you can create the virtual environment anywhere on your system.Optional: Instead of venv, one can also use a miniconda environment. You can download miniconda from here. Note: Anaconda environments are not supported. Create a new Python 3.11 miniconda environment without linking packages from the base environment using
conda create -n spinetoolbox python=3.11
-
Activate the venv environment on Windows (provided that you are in
Spine-Toolbox
directory) usingcd ./.venv/Scripts activate
Or on bash, zsh
source .venv/bin/activate
or on a Conda prompt (if you are using a miniconda environment instead of venv)
conda activate spinetoolbox
Make sure that the terminal prompt now indicates the active environment.
-
Use the
cd ..
command to go back to theSpine-Toolbox
directory (the one that containsrequirements.txt
) if needed. -
Install Spine Toolbox along with its dependencies with
python -m pip install -r requirements.txt
-
(Optional) Install additional development packages with
python -m pip install -r dev-requirements.txt
-
Once the installation has finished, you can now launch Spine Toolbox using
spinetoolbox
To upgrade, pull the latest changes from the Spine Toolbox repository (while in Spine-Toolbox
directory)
git pull
git fetch --tags origin
and run (to upgrade the other Spine Toolbox packages)
python -m pip install -U -r requirements.txt
NOTE: The version number of Spine packages are automatically
generated from Git tags during the python -m pip install ...
step.
This makes it important to fetch new tags from GitHub before running the
python -m pip install ...
when upgrading. This is true also for editable
installations.
If you are alternating between multiple branches, you should repeat
the pip install -e .
step after switching a branch. This ensures
the version number is updated correctly. On the other hand if you do
not need the version number to be precise, it is okay to skip the
step.
To illustrate with an example, consider there are two feature branches
P
and Q
that you are working on. Both are based on master
after
the release 0.6.18
.
0.6.18
---o----o--o--o P
\
o--o--o--o Q
Then the versions are:
P
:0.7.0.dev3+g<commit-hash-P>.d<commit-date-P>
,Q
:0.7.0.dev4+g<commit-hash-Q>.d<commit-date-Q>
.
The version should be available inside Python as:
from spinetoolbox.version import version_tuple
print(version_tuple)
# P: (0, 7, 0, 'dev3', 'g<commit-hash-P>.d<commit-date-P>')
# Q: (0, 7, 0, 'dev4', 'g<commit-hash-Q>.d<commit-date-Q>')
So if your code uses any of the components of the version_tuple
that
might change, you should re-run pip install
, otherwise this can be
ignored safely.
There are old Windows installer packages available for a quick install, but they are at this point (26.1.2023) quite obsolete and cannot be recommended for anything but a quick look at how Spine Toolbox looks and feels (although even that has changed). Download the installer package from here, run it, and follow the instructions to install Spine Toolbox.
Python 3.8.1 or later is required. Python 3.8.0 is not supported due to problems in DLL loading on Windows.
See the files pyproject.toml
and requirements.txt
for packages required to run Spine Toolbox.
(Additional packages needed for development are listed in dev-requirements.txt
.)
The requirements include three packages (spinedb_api
,
spine_engine
, and
spine_items
), developed by the Spine project consortium.
You can find the latest documentation on readthedocs.
If you want to build the documentation yourself,
source files for the User Guide can be found in docs/source
directory. In order to
build the HTML docs, you need to install the development requirements (see section
'About requirements' above). This installs Sphinx (among other things), which
is required in building the documentation. When Sphinx is installed, you can build the
HTML pages from the user guide source files by using the bin/build_doc.bat
script on
Windows or the bin/build_doc.py
script on Linux and Mac. After running the script, the
index page can be found in docs/build/html/index.html
. The User Guide can also
be opened from Spine Toolbox menu Help->User Guide (F1).
The first thing is to make sure that you are not using Anaconda. Only Miniconda is supported. Anaconda's base environment includes Qt related packages and we suspect that they leak some shared Qt libraries into environments, even when specifically requesting that base environment packages should not be linked to environment packages. See also Problems in starting the application below.
Please make sure you are using Python 3.8.1 or later to install the requirements.
If you see the following error when running the command python -m pipx install spinetoolbox
No Python at 'c:\python38\python.exe'
Fatal error from pip prevented installation. Full pip output in file:
Where c:\python38\python.exe may be some other path. To fix this, delete the folder
C:\Users\<user name>\.local\pipx\shared
and run the python -m pipx install spinetoolbox
command again.
If Python runs into errors while installing on Linux systems, running the following command in a terminal may help:
$ sudo apt install libpq-dev
If there are problems in starting Spine Toolbox, the chances are that the required
packages were not installed successfully. In case this happens, the first thing you
should check is that you don't have Qt
, PyQt4
, PyQt5
, PySide
, PySide2
, and
PySide6
packages installed in the same environment. These do not play nice together
and may introduce conflicts. In addition, make sure that you do not have multiple versions
of these Qt
related packages installed in the same environment. The easiest way
to solve this problem is to create a blank (e.g. venv virtual environment) Python
environment just for PySide6
applications and installing the requirements again.
Warning: Using the conda-forge channel for installing the requirements is not recommended.
The required qtconsole
package from the conda-forge channel also
installs qt
and PyQt
packages. Since this is a PySide6
application, those
are not needed and there is a chance of conflicts between the packages.
Note: Python 3.8.0 is not supported. Use Python 3.8.1 or later.
Wednesday Sep 8, 17:00-18:30 CEST (11:00-12:30 EDT) - Organized together with G-PST
Spine Toolbox is an open source software to manage data, scenarios and workflows for modelling and simulation. You can have your local workflow, but work as a team through version control and SQL databases. This webinar gives a quick overview of the different functionalities and showcases them through two examples.
Recording Chapters:
- 00:00-00:05 Relation to G-PST Pillar 5 (Clayton Barrows, NREL)
- 00:05-00:35 Building a workflow in Spine Toolbox PDF, (Juha Kiviluoma, VTT)
- 00:35-00:55 Example workflow from Canada to manage lots of input sources PDF, (Madeleine McPherson and Jake Monroe, University of Victoria)
- 01:05-01:15 Example workflow from EU project TradeRES to serve multiple models (Milos Cvetkovic and Jim Hommes, TU Delft)
- 01:15-01:30 Moderated Q&A
Tuesday Sep 7, 14:00-15:30 CEST (8:00-9:30 EDT) - Organized together with EERA ESI
The importance of operational details in planning future energy systems has been rapidly increasing. One driver for this is the increasing role of variable power generation, which requires that the energy system models consider higher temporal granularity, longer time series and new aspects in power system stability. Another important driver is a consequence of sector coupling through the electrification of transport, space heating and industries. As the other sectors become more integrated with electricity, they need to be modelled at a granularity that is sufficient for describing the potential flexibility they can bring to the power system dominated by variable and uncertain power generation.
This webinar will present the open source Julia based energy system modelling framework SpineOpt, which has been built with these challenges in mind. It can represent different energy sectors using representation typically available only in sector specific models and with highly adaptable temporal and stochastic structures available both for planning and operations. More information at https://spine-project.github.io/SpineOpt.jl/latest/index.html.
Recording Chapters:
- 00:00-00:08 Introduction PDF
- 00:08-00:31 Basic elements of SpineOpt and the flexible spatial structure PDF
- 00:31-00:40 Adaptable temporal and stochastic structures PDF
- 00:50-01:30 Representation of different energy sectors & Attempts to make the model faster PDF
- 01:25-01:35 Q&A + Step-by-step 10 minute demo on how to build a simple model with SpineOpt using Spine Toolbox
Thursday Sep 9, 14:00-16:00 CEST (8:00-10:00 EDT)
Full recording, all presentations slides PDF
Over the past 4 years, the EU project Spine has developed a set of open-source tools for modelling complex energy systems. This webinar demonstrates the Spine software through six selected case studies, covering topics such as sector coupling, co-optimization of operation and investments, stochastic modelling, and rolling horizon optimization. Each subsection described below consists of a brief introduction followed by a live demonstration of the particular case, where some of the outstanding features of Spine are highlighted and discussed.
Recording Chapters:
- 00:05-00:15 Introduction to Spine: This section uses a simple example to demonstrate the SpineOpt modelling principle. First, the user defines the different objects in their system, such as units and nodes, as well as the relationships between them, such as which units are connected to which nodes. Then, they specify values for certain pre-defined parameters such as node demand, unit capacity, cost, and conversion ratio. The resulting dataset is passed to SpineOpt which generates the corresponding optimisation model, optimizes it, and delivers the results.
- 00:15-00:30 Hydro: This section demonstrates hydropower modelling in Spine as performed in Case study A5. The objective is to model part of the Swedish hydropower system, namely the Skellefte river with its 15 power stations, by coupling the river system with the power system. The model maximizes income from electricity supply over one week with an hourly resolution, while respecting basic hydrological constraints.
- 00:29-00:45 Building heating: This section demonstrates building heating modelling in Spine as performed in Case study A4. The objective is to model the Finnish power and district heating system coupled with an electrically heated residential building stock. The result is a rolling unit commitment and economic dispatch model, that optimizes system performance over one year at hourly resolution.
- [00:45-00:57] Break
- 00:57-01:06 Gas grid: This section demonstrates gas grid modelling in Spine as performed in Case study A2. The objective is to model a natural gas transmission system with pressure-driven gas and couple it with an electricity system to capture the flexibility provided by the gas network. The result is a dispatch model that co-optimizes operations in both systems over a day at hourly resolution.
- 01:06-01:22 Stochastic: This section demonstrates stochastic modelling in Spine using a simple example system. Three different stochastic structures are demonstrated: purely deterministic, stochastic fan, and converging fan.
- 01:22-01:35 Power grid investments: This section demonstrates power grid investment modelling in Spine as performed in case study C2. The objective is twofold: (i) to model the Nordic synchronous power system (Norway, Sweden, Finland, and Denmark) with high operational detail; and (ii) to find optimal transmission line investment decisions over a long-term horizon of 10 years, for three different wind penetration scenarios.
- 01:35-01:40 Q&A
Friday Sep 10, 14:00-15:30 CEST (8:00-9:30 EDT)
Full recording, all presentations PDF
Creation of new optimisation models requires a lot of work to get the data to the models and the results out of the models. Spine Toolbox is an open source data and workflow management tool to assist with these tasks and can work with models written in any language. Meanwhile, SpineInterface is a Julia package that links the data management capabilities of Spine Toolbox with Julia/JuMP modelling environment in a very direct way.
The data interfaces of Spine Toolbox together with SpineInterface simplifies the process of developing optimization models by allowing the model developer to focus on the constraint equations. The required data structures and data are defined with a graphical interface in Spine Toolbox and are immediately available to the model developer inside the constraint equation code without any action or code required by the model developer. SpineInterface supports the full range of data parameter types supported by Toolbox and provides a mechanism for representation of time and time-based data, either time series, time patterns or arbitrarily varying temporal data.
This session will be of interest to model developers and/or students who want a significant head start in developing optimization models. The sessions will also be of interest to model developers who may wish to translate existing models developed in other platforms such as GAMS, into the Spine framework using SpineInterface. The power of SpineInterface will be demonstrated in an interactive session where the full modelling workflow will be illustrated from data structure design and implementation to constraint equation development.
Agenda:
- Overview of SpineInterface
- Toolbox concepts and data structures including the Spine data API
- SpineInterface: convenience functions and accessing Spine Toolbox data in Julia
- Defining a model data structure in Spine Toolbox
- Building and solving an optimization model using SpineInterface
- Q&A + live demo 00:45-01:21
All are welcome to contribute!
See detailed instructions for contribution in Spine Toolbox User Guide.
Below are the bare minimum things you need to know.
-
Install the developer requirements:
python -m pip install -r dev-requirements.txt
-
It is also recommended to run
pre-commit install
in the respective root directories of spine-toolbox, spinedb_api, spine-engine and spine-items. This sets up git hooks for auto-formatting the files before committing.
- Black is used for Python code formatting. The project's GitHub page includes instructions on how to integrate Black in IDEs.
- isort is used for formatting the Python import statements.
- Google style docstrings
Running the following line in the project root will take care of sorting the imports:
isort .
It is advisable to run pylint
regularly on files that have been changed.
The project root includes a configuration file for pylint
.
pylint
's user guide includes instructions on how to
integrate the tool in IDEs.
Unit tests are located in the tests
directory.
You can run the entire test suite from project root by
python -m unittest
If you think you have found a bug, please check the following before creating a new issue:
- Make sure you’re on the latest version.
- Try older versions.
- Try upgrading/downgrading the dependencies
- Search the project’s bug/issue tracker to make sure it’s not a known issue.
What to put in your bug report:
- Python version. What version of the Python interpreter are you using? 32-bit or 64-bit?
- OS. What operating system are you on?
- Application Version. Which version or versions of the software are you using? If you have forked the project from Git, which branch and which commit? Otherwise, supply the application version number (Help->About menu).
- How to recreate. How can the developers recreate the bug? A screenshot demonstrating the bug is usually the most helpful thing you can report. Relevant output from the Event Log and debug messages from the console of your run, should also be included.
The developers of Spine Toolbox are happy to hear feature requests or ideas for improving existing functionality. The format for requesting new features is free. Just fill out the required fields on the issue tracker and give a description of the new feature. A picture accompanying the description is a good way to get your idea into development faster. But before you make a new issue, please check that there isn't a related idea already open in the issue tracker.