Ways of further improving

[seovchinnikov]

Hey,
Ive tried your different setups, and I got around 96.7 on my dataset with resnet and nn classifier with l1 distance.
Did you try combine multiple pretrained cnns to get more features for classification step?

[sujitpal]

Hi @seovchinnikov, very cool. No I didn't do that, that might be a good thing to try.

[seovchinnikov]

Also, Im planning to try feature fusion for all layers as in https://arxiv.org/pdf/1604.00133.pdf
For this technique on VGG the feature dimension is calculated as 64 + 128 + 256 + 512 + 512 + 4, 096 + 4, 096 = 9, 664.

[sujitpal]

I will check it out, thanks for the link. Would be great if you let us know how it went, would be good for others (including me) who might want to replicate.

[seovchinnikov]

Ok, thank you, I'll write here about results.
If anyone has questions concerning Keras' code for sampling, pooling and concatenating output from intermediate layers of pretrained models, don't hesitate to contact me.

[sujitpal]

Also @seovchinnikov, would be great if you can put pointers here for your Keras code as well if you are okay with that.

[seovchinnikov]

I've created draft for resnet50 here (hardcoded include_top=True)
https://gist.github.com/seovchinnikov/319303c13fa8db76da4e91705cdc9a97

[seovchinnikov]

I've finished evaluation and it's 98.0 now. So it improved for around 1.3% on my dataset. I think its good result because it doesnt require additional computation of convolutions.

[seovchinnikov]

Also, I would like to try resnet101, resnet152, and apply the same technique

[anilmaddala]

@seovchinnikov I am working on something similar, do you have a notebook where I can test your approach? Also are you using the same INRIA Holiday Photos Dataset?

[seovchinnikov]

@anilmaddala, hey
I've attached modification of original ResNet50 factory function above called ResNet50Fused, so you can import it and use instead of original one.

I did not test it on holiday dataset but on my own dataset it increases accuracy (look at some comments above).

So you need just replace factory function call in the code by ResNet50Fused and train as usual. Please see https://github.com/sujitpal/holiday-similarity/blob/master/src/03-pretrained-nets-vectorizers.ipynb

Please let me know if you'll get numbers on the holiday dataset

[taflahi]

I am using your fused Resnet50 (with dot/elementwise cosine distance) on holiday and it is really increased

But I changed the number of epochs to 30. Maybe can increase more on a greater number of epochs.

[seovchinnikov]

Thats great, be careful though, it may be more prone to overfitting because it contains more low level features

[aihill]

Hi @seovchinnikov thank you very much for posting your code and as @taflahi pointed above it shown a great improvement in accuracy for holiday dataset. Can I use your code keras code for applying it to any other similar applications ?

[seovchinnikov]

I'm glad it helps you, feel free to try it on other datasets (big enough) 👍

[aihill]

Thank you very much @seovchinnikov

[akhil342]

Hi @seovchinnikov @sujitpal ,
I am working on something similar, I would like to try mobilenet as a feature extractor. I don't know how to implement? Can you please guide me?

[kubytskyi]

If you do have problem with
TypeError: _obtain_input_shape() got an unexpected keyword argument 'include_top'

simply change include_top=include_top with require_flatten=include_top in @seovchinnikov 's file resnet_fused.py

it happens due to differences in Keras versions

[jcgarciaca]

@taflahi
Could you tell me how did you get that performance with holiday dataset?
I used ResNet50Fused and trained the network but my results are not good in validation set, as shown in below figure