google-research/text-to-text-transfer-transformer

Score function usage

antonio-mastropaolo opened this issue · 2 comments

Hi everybody,

I have a question about the model.score function:

If I want to retrieve the model's confidence on a given input, does it make sense to feed the model.score function like so?
model.score(inputs=inputs, targets=predictions). Where input is the actual input file, and predictions is the file containing all the predictions obtained previously.
I am assuming that such score will further be normalized between [0, 1] range in order to retrieve the final confidence

Yes, that will work, but it will give you the model's assigned log-likelihood, not a confidence between 0 and 1.

(this is not an issue - please open a discussion for questions like this)