Weights Initialization

[liyi-ff]

When initializing weights to the same value, the weights will also be the same during the training.
(

adapt-image-models/mmaction/models/backbones/vit_clip.py

Line 182 in 2880777

for n, m in self.transformer.named_modules():

)

Is there any reason to initialize this to a constant 0 instead of other initialization? For example, Gaussian with a mean of 0 and a very small variance.

Thanks

[taoyang1122]

Hi @liyi-ff , we initialize it to zero so that at the beginning of training, the model is the same as the pre-trained model. Then with the training process, the adapters gradually adapt the model to video data. This stabalizes the training process.