Exception - You are attempting to share a same 'BatchNormalization' layer

[edwarner]

Hello,

Great Q-learning example!

I'm getting the following error running on latest dev builds of Theano and Keras.

D:\Development\DeepLearning\QLearning4k\examples>python test_snake.py
Using Theano backend.
Using gpu device 0: GeForce GTX 970 (CNMeM is disabled, cuDNN 4007)
Epoch 001/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 0
Epoch 002/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 0
Epoch 003/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 0
Epoch 004/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 1
Epoch 005/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 1
Epoch 006/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 1
Epoch 007/10000 | Loss 0.0000 | Epsilon 1.00 | Win count 1
Traceback (most recent call last):
File "test_snake.py", line 24, in
agent.train(snake, batch_size=64, nb_epoch=10000, gamma=0.8)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\qlearning4k\agent.py", line 91, in train
batch = self.memory.get_batch(model=model, batch_size=batch_size, gamma=gamma)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\qlearning4k\memory.py", line 35, in get_batch
return self.get_batch_fast(model, batch_size, gamma)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\qlearning4k\memory.py", line 106, in get_batch_fast
self.set_batch_function(model, self.input_shape, batch_size, model.output_shape[-1], gamma)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\qlearning4k\memory.py", line 88, in set_batch_function
Y = model(X)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\keras\engine\topology.py", line 500, in call
return self.call(x, mask)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\keras\models.py", line 164, in call
return self.model.call(x, mask)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\keras\engine\topology.py", line 1951, in call
output_tensors, output_masks, output_shapes = self.run_internal_graph(inputs, masks)
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\keras\engine\topology.py", line 2093, in run_internal_graph
output_tensors = to_list(layer.call(computed_tensor, computed_mask))
File "C:\WinPython34\python-3.4.4.amd64\lib\site-packages\keras\layers\normalization.py", line 116, in call
raise Exception('You are attempting to share a '
Exception: You are attempting to share a same BatchNormalization layer across different data flows. This is not possible. You should use mode=2 in BatchNormalization, which has a similar behavior but is shareable (see docs for a description of the behavior).

Changing to mode=2 produces another error...

[farizrahman4u]

There are 2 ways to get around this bug for now:

• Remove the BatchNormalization layer
• set fast=False for EperienceReplay

[edwarner]

Thank you.

Removing the batch normalization layer causes training to freeze during or after epoch 5.

Setting fast=False seems to work.