How to fine-tune a new task？

[Binyun-Z]

I want to fine-tune a new task for protein classification，using “ bert-base ”pretrained model，But the classification accuracy is only 0.5.This means that my model is not learning new parameters,canyou tell me why?

class Tape_Classifier(nn.Module):
    def __init__(self, pretrain_model, config):
        super().__init__()
        self.pretrain_model = pretrain_model
        self.classifier = nn.Sequential(
                            nn.Linear(config.hidden_size1, config.hidden_size2),
                            nn.Linear(config.hidden_size2,config.num_classes))    
    def forward(self, input_ids):
        outputs = self.pretrain_model(input_ids)
        
        output = self.classifier(outputs[0]) # batch, hidden
        return output
pretrain_model = torch.load('../model/bert-base.pt')
pretrain_model.train()
class config:
    hidden_size1 = 768
    hidden_size2 = 256
    num_classes   = 2
model_tape_bert = Tape_Classifier(pretrain_model,config)

[rmrao]

Sorry I don't think I can really help debug this without the full training script you're using, and if it's a custom task I don't know that I'll be able to help - there are way too many possibilities. If you think there's a specific bug in our code, please open a new issue.