pplonski/keras2cpp

Support for multi-input and merge layer?

blackarrow3542 opened this issue · 12 comments

blackarrow3542 commented

Hi, I find this project very useful and interesting! Thanks a lot!
Will you add support for multi-input and merge layer?

Thanks

pplonski commented

Hi, yes I want to support multi-input and merge layer. Do you have an example of Keras network that you want to use, a reference would be great.

spurihwr commented

Hi, Thanks for the project. I find it quite useful as I am trying to deploy keras model in cpp.
I came across a small bug in your keras_model.cc file in function:
std::vector< std::vector > keras::conv_single_depth_valid(
std::vector< std::vector > const & im,
std::vector< std::vector > const & k)
{}

In the nested loop, a 2 is missing in front of st_x and st_y . It should be something like:
for(unsigned int i = st_x; i < im.size()-2st_x; ++i) {
for(unsigned int j = st_y; j < im[0].size()-2st_y; ++j) { ....

Thank you so much. I was able to get the correct prediction by changing this.

pplonski commented

Hi @spurihwr ! In conv_single_depth_valid the st_x and st_y values are borders, so you need to add one border at start and one at end, something like thisfor(unsigned int i = st_x; i < im.size()-st_x; ++i) - this is strange, that if you changed this you started to get correct predictions. I'm going to add better testing for dumping, so I'll look closer on in. Could you share your network for testing? (is it large?)

spurihwr commented

Hi @pplonski !! Thanks for your reply.
Actually, with your original code, there was a segmentation fault and that is because the loop was accessing outside the size of image in
y[i - st_x][j - st_y] += k[k.size() - k1 - 1][k[0].size() - k2 - 1] * im[i - st_x + k1][j - st_y + k2];
With im.size() = 8 and kernal size k1=k2=4, im[i-st_x+k1] goes out of range at i=6.
My network is something like this:
model = Sequential()
model.add(Convolution2D(4, 3, 3, border_mode='valid', input_shape=(1,8,8)))
model.add(Activation('relu'))
model.add(Convolution2D(4, 3, 3, border_mode='valid'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

pplonski commented

for kernel size = 4, indeed there can be a problem but kernels are odd (usually, I haven't seen even kernel size ??). So for kernel size = 3, st_ will be 1, and for i=6, im[i-st_x+k1] will be working, and for kernel size = 5, st_x will be 2, and i=6 will be not considered (because border mode is valid). Does it make sense?

spurihwr commented

Yes. I was initially using a kernal size of 4 which created this problem. As I am new to CNN, I was not aware of this that kernal should be odd. Thank you so much. The code is running fine. I want to use Conv1D for some task. So i will try to implement that as I see that it is not in the code and will update you here abt that. Thanks :)

xdtl commented

Do you have any suggestions for supporting multi-input and merge layer? Thanks!

blackarrow3542 commented

@xdtl Hi, since keras2cpp doesn't support some neural network models, an alternative solution is to build Tensorflow with makefile to implement a Tensorflow model in an existing c++ project.
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/makefile

xdtl commented

Thanks for your reply! Does it support Windows?

xdtl commented

Thanks! I will look into this...
It would still be great if there is a light weighted framework that can deploy trained Keras models for prediction.