Reservoir computing with single dynamical node

In a 2011 article titled "Information processing using a single dynamical node as complex system" L. Appeltant et al. showed that a nonlinear node with delayed feedback, such as a Mackey-Glass oscillator, can be used in lieu of a complex network in the paradigm of reservoir computing. The efficacy of their approach was confirmed through the NARMA10 benchmark: Four NARMA10 samples with length 800 points each, were preprocessed and fed through the single-node reservoir and the reservoir response was used for training and testing. A Normalized Root Mean Square Error (NMRSE) down to 0.12 (with twofold cross-validation) was achieved.

The present repository reproduces this result. It contains the following scripts:

The Narma10_Input.py script generates a Narma10 file to be used as input for the single-node reservoir. This file contains the multiplexed 'masked' signals which are fed to the Mackey-Glass Oscillator(s). A Narma10.h5 file that yields the desired cross-validation NMRSE = 0.12 is included. The Narma10_Input.py can be used to generate alternate Narma10 files.
The MG_Oscillator.py runs the Mackey-Glass simulation with the relevant input file. The oscillator response is saved in a file that is then used for training the network.
The Training.py script uses the Mackey-Glass oscillator response to train the network and calculate the NMRSE.
The Visualization.py script generates the figures shown here: (a) the response of the reservoir; (b) the fit on the training set and (c) the fit on the test set.
The DynamicalNode_Reservoir.py is a wrapper for the above scripts--the main program.

Notes:

on the Narma10 input

The binary masks used for multiplexing follow the guidance in this (2014) paper by L. Appeltant et al.

on the Mackey-Glass solver

The standard form of the Mackey-Glass equation is:

This equation appears in Appeltant et al. (2011) with a different notation:

where clearly we recover the previous expression for:

For the numerical integration of this equation we use the implicit midpoint method as described below:

where h is the timestep. To integrate a Delay Differential Equation (DDE) we need the values of X at the present time X(t) as well as its delayed value X(t-τ). For the implicit midpoint method we actually need X(t_n) and X(t_n+h/2-τ). Therefore, if τ-h/2 is an integral number N of timesteps h, viz. if τ-h/2 = Nh, then the value of X(t_n+h/2-τ) has already been calculated and we can immediately substitute it in the equation above. We choose:

where N represents the number of steps used to resolve the Delay Period τ.

on the Mackey-Glass response

The response of the reservoir along with the input is shown for the normalized time interval [0,100]. The delay τ was chosen τ = 80 such that a transition at t = τ = 80 can be observed in the input signal from the first Narma10 value to the second. Each value from the Narma10 dataset is multiplexed through 400 virtual nodes with temporal distance ι = τ/400 = 0.2. For an 800-long Narma10 signal the maximum normalized time is 800*80 = 64000.

on training

The fit is shown on the training set (samples 0 and 1).

The fit is shown on the test set (samples 2 and 3).