ymcui/Chinese-BERT-wwm

关于不能用RobertaModel引用

Jackthebighead opened this issue · 8 comments

您好,在readme中的快速加载有这样一句:

注意:本目录中的所有模型均使用BertTokenizer以及BertModel加载,请勿使用RobertaTokenizer/RobertaModel!

我从huggingface官网下下来了roberta-chinese-wwm的pytorch_model.bin,tokenizer.json和vocab,再想在代码使用.from_pretrained()方法引用,即不是采用快速加载的情况,请问这样的话是用RobertaModel引用还是BertModel引用呢?我尝试用BertModel引用,但是提示:

You are using a model of type roberta to instantiate a model of type bert. This is not supported for all configurations of models and can yield errors.`

还请解答一下,非常感谢!

如果你的from_pretrained放的是下载的路径的话,就是bertmodel引用robertamodel的预训练参数
不过这不影响使用,因为bertmodel和robertamodel的微调部分代码是一样的

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

如果你的from_pretrained放的是下载的路径的话,就是bertmodel引用robertamodel的预训练参数 不过这不影响使用,因为bertmodel和robertamodel的微调部分代码是一样的

明白了,感谢。

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale commented

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.

老哥,你好,请问这个问题不需要解决吗 因为我在使用dietilbert也遇到同样的问题