Add channels from 3 to 4
Closed this issue · 5 comments
Thank you for your good work trzy.
I want to use your faster r-cnn programs for my task.
And I want to add channels from 3 (RGB) to 4 (RGB + New one). Where should I change the program?
Thank you.
I will use pytorch version. Then, would you teach me that version?
This isn't a simple modification. You would at minimum have to change the first layer of the backbone to accept such an input. But then the current backbone weights will no longer apply. You could potentially un-freeze the first layer for training and randomly initialize it but my guess is that this won't work because the subsequent layers have been learned to embed RGB data processed by the original layer. Un-freezing all the layers and training them will probably not work well at all, either.
I think you'd need to retrain the backbone (VGG-16 or ResNet) in a way that utilizes this additional data. The backbones are trained on ImageNet, which if I remember correctly includes 14 million images. If you have a sufficiently large 4-channel data set, you could train a new backbone and then use that.
Thank you for your fast reply.
You mean that a model which include 4 channels can be built by training backbone from scrach, right?
(I should not use previous training models.)
Yup. You would need to modify the first layer (the first convolutional filter) whose output influences all subsequent layers. Therefore, the entire backbone needs to be retrained to make use of this added information. You cannot just change the size of the network layers without retraining.
OK. I will try your sugesstions.
Thank you very much.
I will close this issue. I may ask something again.