ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.
Opened this issue · 4 comments
Can you give me some advice about this Error?
2019-07-31 08:51:00,403 - algorithms.Algorithm - INFO - Training: miniImageNet_Conv32CosineClassifier
20%|█▉ | 197/1000 [00:05<00:21, 37.46it/s]2019-07-31 08:51:06,434 - algorithms.Algorithm
100%|██████████| 1000/1000 [00:27<00:00, 35.99it/s]
2019-07-31 08:51:28,193 - algorithms.Algorithm - INFO - ==> Training stats: {'loss': 2.469}
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO - Evaluating: miniImageNet_Conv32CosineClassifier
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO - ==> Dataset: MiniImageNet_val [2000 batches]
0%| | 0/2000 [00:00<?, ?it/s]/data1/zjj/meta-code/withoutForgetting/algorithms/FewShot.py:185: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
images_test_var = Variable(images_test, volatile=is_volatile)
/data1/withoutForgetting/algorithms/FewShot.py:190: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
images_train_var = Variable(images_train, volatile=is_volatile)
100%|██████████| 2000/2000 [01:14<00:00, 26.36it/s]
2019-07-31 08:52:43,083 - algorithms.Algorithm - INFO - ==> Results: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
2019-07-31 08:52:43,084 - algorithms.Algorithm - INFO - ==> Evaluation stats: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
Traceback (most recent call last):
File "train.py", line 110, in <module>
algorithm.solve(dloader_train, dloader_test)
File "/data1/withoutForgetting/algorithms/Algorithm.py", line 288, in solve
self.keep_record_of_best_model(eval_stats, self.curr_epoch)
File "/data1/withoutForgetting/algorithms/Algorithm.py", line 359, in keep_record_of_best_model
.format(metric_name))
ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.
The error occurred in:
def keep_record_of_best_model(self, eval_stats, current_epoch):
if self.keep_best_model_metric_name is not None:
metric_name = self.keep_best_model_metric_name
if (metric_name not in eval_stats):
raise ValueError('The provided metric {0} for keeping the best '
'model is not computed by the evaluation routine.'
.format(metric_name))
metric_val = eval_stats[metric_name]
Thanks a lot!
change back to pytorch version==0.4
This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line:
https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20
Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.
This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line:
https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20
Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.
nice!!!
This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line: https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20 Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.
Hi, bro, i use the same pytorch version like u, but i still get the error ,could you help me ?