Yubi2Community/YubiAI

Excellent architecture, however key error clf_head_name with classification

Closed this issue · 2 comments

g0dEngineer commented

I was trying some random inputText text value, while using micro-e4 pretrained model, getting error while invoking the following

...

inputText="estimated GPD of America in 2023"

print(get_results(inputText, "clf_head_name"))

Error-end: KeyError: 'clf_head_name'

  • Tried to use language_detection from "outprob = self.model.predict("language_detection", tokens[:510])"

  • ...that I found in "yubiai\nlp\language_detection\yubiLanguageDetection .py", but with no luck, same KeyError

What is the appropriate head name to use for the e4 predictor?

swapaj commented

Hi @g0dEngineer

micro-e4 model is a pretrained model based on RoBERTa architecture. It is a barebone model and just trained on missing-word/token objectives. It does not have a classification head. There is no direct use-case of this model except you get tokens, word/sentence embeddings and fill-in-the-blanks. You can use this model to train further tasks like classification, ner etc etc.

yulan model is fine-tuned version of micro-e4 or small-e8 models. So we added a classification head while fine-tuning and then we use the same name again while retrieving back results.

If you want to train your own model please check these two files for more details finetuning shell script and sample python code

Hope this helps clarify the doubt.

g0dEngineer commented

Thanks, I had found the fine tuning scripts.
I will use the yulan for classification for now.

  • 👍1