Pre-trained weights incompatible with backbone
LucFrachon opened this issue · 2 comments
LucFrachon commented
The weights for ConvNeXt-V2-Base, pretrained on INet1k, provided here, have several incompatibilities with the encoder architecture:
- The MinkowskiLayerNorm layers have a
self.ln
element that means that the checkpoint keys should look likedownsample_layers.1.0.ln.bias
, whereas the weights provided havedownsample_layers.1.0.bias
. - Similarly, all the layers named
pwconv<i>
haveself.linear
, which the checkpoints doesn't have. - The MinkowskiConvolution layers have a parameter
kernel
(e.g.,downsample_layers.1.1.kernel
), whereas the provided checkpoint has keys"downsample_layers.1.1.weight
. - The MinkowskiConvolution biases have shape (1, c) (e.g., (1, 256)) in the model but shape (c,) in the checkpoint, and weights have shapes (p**2, in, out) in the model but (out, in, p, p) in the checkpoint
- Same problem with the DW convolution weights and biases in
stages
: biases need to be (1, c) but are (c,), and weights need to be (p ** 2, c) but are (c, -1, p, p)
These can all be fixed with some code, but it would make life easier for everyone if you could upload the correct weights.
Thanks a lot!
blackpearl1022 commented
@LucFrachon Same idea, I have.
Do you have any updates on your side ?
Thanks !
LucFrachon commented
I wrote code to update the state dict and it worked.
I haven't checked if the provided checkpoints have been updated. I've moved on to other things now...