txsun1997/Black-Box-Tuning

Comparison with gradient-based methods on large models

pretidav opened this issue · 0 comments

It would be very useful a comparison with performances of gradient-based methods (lora,p-tuning,prompt tuning, etc.) on the same datasets and using the same models (i.e., t5-xxl) commonly used in literature. For instance you can compare with results quoted here https://aclanthology.org/2021.emnlp-main.243.pdf .
It is not clear from your manuscript whether or not the proposed approach is still competitive with (very) large models (larger than roberta-large), where it is well known that gradient-based models are performing very well.

Thank you, and congratulation for the very very interesting method!