aigamedev/scikit-neuralnetwork

Top 5 error is always zero? Matconvnet

AlsubaieNajah opened this issue · 2 comments

Hi,
I am using MatConvNet-beta18. I have 5 class each class has 34 sample (divided into training and testing group).
The problem is that my top5 error is always zero? Does anyone have any idea why I am getting this and how to solve it. Thank you.

This is sample output:
train: epoch 01: 1/ 2: 7.4 Hz obj:95.7 top1err:0.73 top5err:0 [100/100]
train: epoch 01: 2/ 2: 9.0 Hz obj:87.4 top1err:0.706 top5err:0 [36/36]
val: epoch 01: 1/ 1: 32.8 Hz obj:20.6 top1err:0.735 top5err:0 [34/34]
cnn_train: saving model for epoch 1
cnn_train: model saved in 0.00021 s
train: epoch 02: 1/ 2: 21.2 Hz obj:73.6 top1err:0.75 top5err:0 [100/100]
train: epoch 02: 2/ 2: 23.2 Hz obj:83.8 top1err:0.75 top5err:0 [36/36]
val: epoch 02: 1/ 1: 35.1 Hz obj:66.3 top1err:0.853 top5err:0 [34/34]
cnn_train: saving model for epoch 2
cnn_train: model saved in 3e-05 s
train: epoch 03: 1/ 2: 22.8 Hz obj:90.9 top1err:0.71 top5err:0 [100/100]
train: epoch 03: 2/ 2: 24.5 Hz obj:93.4 top1err:0.684 top5err:0 [36/36]
val: epoch 03: 1/ 1: 39.9 Hz obj:39.3 top1err:0.5 top5err:0 [34/34]
cnn_train: saving model for epoch 3
cnn_train: model saved in 3.2e-05 s
train: epoch 04: 1/ 2: 13.8 Hz obj:86.2 top1err:0.64 top5err:0 [100/100]
train: epoch 04: 2/ 2: 14.4 Hz obj:89.6 top1err:0.632 top5err:0 [36/36]
val: epoch 04: 1/ 1: 24.4 Hz obj:38.6 top1err:0.676 top5err:0 [34/34]
cnn_train: saving model for epoch 4
cnn_train: model saved in 3.8e-05 s
train: epoch 05: 1/ 2: 22.3 Hz obj:75.8 top1err:0.6 top5err:0 [100/100]
train: epoch 05: 2/ 2: 24.0 Hz obj:80.2 top1err:0.588 top5err:0 [36/36]
val: epoch 05: 1/ 1: 28.4 Hz obj:49.9 top1err:0.559 top5err:0 [34/34]
cnn_train: saving model for epoch 5
cnn_train: model saved in 3e-05 s
train: epoch 06: 1/ 2: 19.7 Hz obj:82.6 top1err:0.68 top5err:0 [100/100]
train: epoch 06: 2/ 2: 21.2 Hz obj:72.5 top1err:0.632 top5err:0 [36/36]
val: epoch 06: 1/ 1: 26.5 Hz obj:28.8 top1err:0.382 top5err:0 [34/34]

and it goes on similarly.. .

Hi,
To explain the problem, In file cnn_train and inside the function error_multiclass. I noticed that I always get zero for this term (min(error(:,:,1:m,:),[],3))). The reason is because the computed error for each class is either 1 or zero and the minimum becomes zero.

error = ~bsxfun(@eq, predictions, labels) ;
err(1,1) = sum(sum(sum(mass .* error(:,:,1,:)))) ;
err(2,1) = sum(sum(sum(mass .* min(error(:,:,1:m,:),[],3)))) ;

Any help is appreciated.

Regards,

With five classes you will always have a top5err of zero, as you will always have a top10err of zero with ten classes etc.