Sequence to Sequence Bert Classification Model

[markpb2-ai]

I have been running into an issue where I load the pre-trained sequence to sequence classification model and then use the model for a classification. However, if I reload the same model and rerun the exact same classification, I get completely different output. I investigated the different layers of the two separately loaded models, and they are the same apart from one layer. I am not exactly sure how to get reliable output from the model if the parameters of this layer are different each time I load them. Thanks for the help!

[rmrao]

We do not provide pretrained weights for the supervised downstream tasks. If you want a model for this, you should be able to train your own secondary structure prediction model using the code + data provided. Right now, the from_pretrained function is only loading weights for the pretrained portion of the model and the supervised head is randomly initialized. I believe it should even display a note saying that certain keys are missing (although this may depend on your python logging level).