About colouring
Closed this issue · 4 comments
Hi,
Thanks for this exciting and excellent work!
From Table.3, we can indicate that coloring is essential for tuning the model.
I want to ask how to ensure the learned "lightweight 2D neural network gΦ" can predict RGB values. The predicted value of this model is might not in [0,255].
Or, can we refer to this model as an "Adapter" (an inserted module in the image-model for tuning tiny-sized parameters).
Looking forward to your insights！
Hi, thanks for your interest in our work!
About the coloring module, the Sigmoid function in this line of models/layers/encoder.py ensures that the pixel values of the predicted image are within [0,1] interval. We then use IMAGENET_DEFAULT_MEAN and IMAGENET_DEFAULT_STD to normalize the image, following the common practice of processing a natural image in 2D domain.
Secondly, we don't think that our coloring module is an "Adapter". To the best of my knowledge, an "Adapter" is a task-specific module that is inserted in the pre-trained model and directly changes intermediate features. However, the proposed P2P module produces the input to the image model (an image) and doesn't affect its architecture design. We think it is a "Prompting" module, which converts downstream task (point cloud analysis under this condition) to pre-trained task (image classification) to leverage the pre-trained knowledge.
Yes. I agree.
Then very interesting to know how D + C (Fig.3) performs. What do you think?
Have you done such experiments?
That's an interesting idea. However, we haven't tried. Our paper mainly focuses on proposing a new prompting mechanism, thus ablations only compare P2P prompting with other prompting method like VPT.
I'm looking forward to see how well combining these two prompting would perform.
ZhangYuanhan-AI
That's an interesting idea. However, we haven't tried. Our paper mainly focuses on proposing a new prompting mechanism, thus ablations only compare P2P prompting with other prompting method like VPT.
I'm looking forward to see how well combining these two prompting would perform.
Thanks!