Validate multi-clusters-sim version simulator

[fumik]

The purpose is to confirm that the multi-clusters-sim version simulator generates the identical results created by master version simulator.

[fumik]

Because the random number generator of the multi-clusters-sim version simulator generates different sequence of random numbers, I modified the multi-clusters-sim version simulator so that it generates exactly identical random numbers of master version of simulator. This validation version of source codes reside in multi-clusters-sim-validation branch. Since the difference between this validation version and multi-clusters-sim version simulator is only random numbers generation logic, I use this validation version simulator for the validation between multi-clusters-sim and master version.

Modifications:

1. Use the same initialization parameters of the random numbers generator.
2. Only the first cluster (clusterID = 0) generates random numbers. Other clusters copy an appropriate portion of random numbers from cluster 0.
3. This version of the simulator is for only validation use, because there are some limitations: maximum number of neurons is 10,000, slower than normal version.

Method:

1. Run master and validation version simulator with test-small-connected.xml configuration file (10*10 neurons, 100x20 epochs, start radius 2.0, tR = 1.9, fE = 0.98) changing number of clusters.
2. Run master and validation version simulator with static_izh_1000.xml configuration file (40*25 neurons, 1x1 epoch, modify StaticConnectionsWeight parameter as min and max are the same values because to offset the difference of synaptic weights - master version simulator uses host random number generator to make the variation of synaptic weights while the validation version simulator uses GPU random number generator) changing number of clusters.

Results:
Got identical results between master and validation version simulator.

Note:

1. We need to slightly modify ConnStatic::setupConnections() function in master version to match the correspondent GPU function in multi-cluster-sim version. (calc distance based on source neurons -> calc distance based on destination neurons)
2. All exe() functions in kernel or device function in master code may need to be casting (double) to generate exactly identical results.

[fumik]

Result images of static_izh_1000.xml configuration file.

Master

Validation 1 GPU

Validation 2 GPUs

Validation 4 GPUs

[stiber]

Looks good. I'll create a new issue to incorporate conditional compilation to select validation vs. production versions of BrainGrid. That may be useful, in the future, for more than just selecting how random numbers are generated.

[fumik]

Finally run master and multi-clusters-sim (not validation) version simulator with tR_1.0--fE_0.98_10000.xml configuration file (100*100 neurons, 100x600 epochs, tR = 1.0, fE = 0.98) with 4 clusters.

The execution times are:
Master version -- 170277s (47.3h)
Multi-cluster-sim w/4 GPUs -- 106887s (29.7h)

Master

Multi-clusters-sim w/4 GPUs

We got qualitatively similar results.

[fumik]

I also run 2 simulations on CPU version of master and multi-clusters-sim.

1. Run master and multi-clusters-sim version simulator with test-small-connected.xml configuration file (10*10 neurons, 100x2 epochs, start radius 2.0, tR = 1.9, fE = 0.98, number of cluster = 1).
2. Run master and multi-clusters-sim version simulator with static_izh_1000.xml configuration file (40*25 neurons, 1x1 epoch, number of cluster = 1).

Results:
Got identical results between master and multi-clusters-sim version simulator.