Will the methods work on other CNNs?

[dashidhy]

Hello authors,

I'm so excited to see that MAE can also work on ConvNets! I was wondering that are the sparseconv MAE and GRN architecture-agnostic that they can also work on other CNNs like a vanilla ResNet without the ConvNeXt tricks? Have you done any experiments on this? Hope you can share some insights.

Thanks!

[shwoo93]

The proposed FCMAE is not limited to a specific convolutional model. While we have not tried it on other architectures, it will likely provide a meaningful initialization typically.

In contrast, the GRN is initially designed to correct the feature collapsing issue observed in the original ConvNeXt model after the FCMAE training. The impact of GRN on other architectures is to be studied.

[Vincent630]

hi,have you try any other convnet，like resnet ，mobilenet？any tips for me ?