Test score is inf

[akshathaarodi]

Hi,

I am able to set up and train the model successfully, but the test scores are inf and the best model is not picked up. How can I fix this one?

EDIT: I suspect it is because of early stopping. But Trn 4 has a better F1 score on val set.

[sattree]

@akshathaarodi Hi Akshath, is this still an issue?

[akshathaarodi]

Yes!

[sattree]

inf is probably just the default value showing up. Wondering why evaluation is being bypassed. Some of the conditionals are likely evaluating to false.

What patience and epoch values are you using? If patience is 3, then why did training terminate after 5 epochs?

[akshathaarodi]

I am using default values. I reduced the train_batch_size to help run on a single GPU.

            train_batch_size=3,
            eval_batch_size=32,
            learning_rate=4e-6,
            num_train_epochs=20,
            patience=3

Not sure how to debug this one.

[sattree]

So, the execution looks fine. Val score was lowest for epoch #1 and training terminated 3 epochs after that. The final scores are correctly based on the checkpoint corresponding to epoch #1. It's been a while but I'll take a look to see why infs are showing up.

As for you original question, the termination criteria is based on val 'score'. 'score' here refers to log loss which was the primary metric of performance. F1 scores were computed to gain additional insights.

Does this answer your question? Are there any other problems you see?

[akshathaarodi]

I see. I was looking at F1 score. Yes, thank you very much!