Seya needs to be updated for Keras 0.2.0
daquang opened this issue · 14 comments
For example, the Bidirectional layer needs to define output_dim now. Also, layers such as Dense no longer have input_dim as a required argument, and the first argument now corresponds to the output_dim
I believe I covered almost everything with this commit 71297e4
Let me know if something was left out
I have tried to use the Bidirectional example with IMDB, and it fails with some theano problem. I've checked it both with the latest keras version and the version tagged as 0.2.0, same result:
Loading data...
20000 train sequences
5000 test sequences
Pad sequences (samples x time)
X_train shape: (20000, 100)
X_test shape: (5000, 100)
Build model...
Train...
Train on 20000 samples, validate on 5000 samples
Epoch 1/4
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/home/beka/dev/seya/examples/imdb_brnn.py in <module>()
66
67 print("Train...")
---> 68 model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=4, validation_data=(X_test, y_test), show_accuracy=True)
69 score, acc = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True)
70 print('Test score:', score)
/home/beka/dev/keras/keras/models.py in fit(self, X, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, show_accuracy, class_weight, sample_weight)
492 verbose=verbose, callbacks=callbacks,
493 val_f=val_f, val_ins=val_ins,
--> 494 shuffle=shuffle, metrics=metrics)
495
496 def predict(self, X, batch_size=128, verbose=0):
/home/beka/dev/keras/keras/models.py in _fit(self, f, ins, out_labels, batch_size, nb_epoch, verbose, callbacks, val_f, val_ins, shuffle, metrics)
213 batch_logs['size'] = len(batch_ids)
214 callbacks.on_batch_begin(batch_index, batch_logs)
--> 215 outs = f(*ins_batch)
216 if type(outs) != list:
217 outs = [outs]
/home/beka/venv/eva/lib/python2.7/site-packages/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
604 self.fn.nodes[self.fn.position_of_error],
605 self.fn.thunks[self.fn.position_of_error],
--> 606 storage_map=self.fn.storage_map)
607 else:
608 # For the c linker We don't have access from
/home/beka/venv/eva/lib/python2.7/site-packages/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
593 t0_fn = time.time()
594 try:
--> 595 outputs = self.fn()
596 except Exception:
597 if hasattr(self.fn, 'position_of_error'):
ValueError: Input dimension mis-match. (input[0].shape[1] = 128, input[1].shape[1] = 64)
Apply node that caused the error: Elemwise{mul,no_inplace}(Join.0, Elemwise{Composite{Cast{float64}(LT(i0, i1))}}.0)
Inputs types: [TensorType(float64, matrix), TensorType(float64, matrix)]
Inputs shapes: [(32, 128), (32, 64)]
Inputs strides: [(1024, 8), (512, 8)]
Inputs values: ['not shown', 'not shown']
Backtrace when the node is created:
File "/home/beka/dev/keras/keras/layers/core.py", line 416, in get_output
X *= self.srng.binomial(X.shape, p=retain_prob, dtype=theano.config.floatX)
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
@amitbeka this seem to be an error I had fixed, the output shape was half of the correct value... also, that example is working for me. Can you please check which commit of Seya are you in? Use this code inside Seya folder
git log --pretty=format:'%h' -n 1
I think I'm using the latest commit: 0b8fd60
in keras i'm on commit 47071f9 (also latest).
tried to update theano to the latest version but that didn't help either
I just ran the experiment in my computer with success. Do you mind creating a new virtual env and installing keras+seya to double check? I honestly think it should be working.
This error was caused because the output_dim being passed by the shape inference was half the actual size. But I fixed that...
just did it (new virtualenv with pip install git+git://github.com/fchollet/keras and manually git clone seya), and still the same result.
I think it has something to do with python versions... The above error happens on python2.7.10.
I tried also a virtualenv with python 3.5, and got a different error:
PYTHONPATH=. python examples/imdb_brnn.py
Loading data...
20000 train sequences
5000 test sequences
Pad sequences (samples x time)
X_train shape: (20000, 100)
X_test shape: (5000, 100)
Build model...
Traceback (most recent call last):
File "examples/imdb_brnn.py", line 65, in <module>
model.compile(loss='binary_crossentropy', optimizer='adam', class_mode="binary")
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/models.py", line 358, in compile
self.y_train = self.get_output(train=True)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/containers.py", line 74, in get_output
return self.layers[-1].get_output(train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 439, in get_output
X = self.get_input(train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 102, in get_input
return self.previous.get_output(train=train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 613, in get_output
X = self.get_input(train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 102, in get_input
return self.previous.get_output(train=train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 412, in get_output
X = self.get_input(train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/core.py", line 102, in get_input
return self.previous.get_output(train=train)
File "/tmp/seya/seya/layers/recurrent.py", line 71, in get_output
Xf = self.forward.get_output(train)
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/layers/recurrent.py", line 446, in get_output
T.unbroadcast(alloc_zeros_matrix(X.shape[1], self.output_dim), 1),
File "/home/beka/venv/seya/lib/python3.5/site-packages/keras/utils/theano_utils.py", line 28, in alloc_zeros_matrix
return T.alloc(np.cast[theano.config.floatX](0.), *dims)
File "/home/beka/venv/seya/lib/python3.5/site-packages/theano/tensor/basic.py", line 2546, in __call__
ret = super(Alloc, self).__call__(val, *shapes, **kwargs)
File "/home/beka/venv/seya/lib/python3.5/site-packages/theano/gof/op.py", line 507, in __call__
node = self.make_node(*inputs, **kwargs)
File "/home/beka/venv/seya/lib/python3.5/site-packages/theano/tensor/basic.py", line 2424, in make_node
(i, s_as_str))
TypeError: Shape arguments to Alloc must be integers, but argument 1 is not for apply node: TensorConstant{64.0}
And from what i see, there is still a small error with the output_dim variable, on recurrent.py, line 26: self.output_dim = self.forward.output_dim
which should be self.forward.output_dim*2 if i'm not mistaken...
Thanks for the quick response (and open-sourcing your work :))!
@amitbeka tkx for taking your time to report your results.
I see that ouput_dim is wrong, but note that the output_shape was actually calculated as https://github.com/EderSantana/seya/blob/master/seya/layers/recurrent.py#L60-L68
Also your python3 error is probably because of the true division. That can be fixed easily.
Anyway, I just made a commit that hopefully will get you covered bb88053
Unfortunately, it only makes python3 error the same as python2, so I still have a dimension mis-match error when running fit().
Does it happen in your setup if you fresh-install a new environment? I wonder what's happening here
@amitbeka I was able to reproduce the error with a clean conda env. I'll check carefully what is happening. If you find out anything, please let me know.
@amitbeka the problem was with the old Theano that conda provides.
Try update Theano as:
$ conda (or pip) uninstall theano
$ pip install git+https://github.com/Theano/Theano.git@15c90dd3#egg=Theano==0.8.git
I'm gonna update Seya requirements.
I just found out that with theano_mode=FAST_COMPILE it doesn't happen, so probably theano. I will validate in the morning. thanks for all the help!
I think it is something with T.concatenate.
Just validated it, and it works great, also in more complex settings (masking). Thanks for the help again
tkx for the feedback @amitbeka