Results are not consistent with the paper

[pvoigtlaender]

Hi,

I just ran the global_table.m script and got the following table

mcg sf-lab sf-motnlc cvos trc msg key sal fst tsp sea hvs jmp fcp bvs ofl
J mean 0.736 0.176 0.540 0.701 0.535 0.520 0.550 0.617 0.447 0.586 0.383 0.591 0.629 0.632 0.662 0.709 0.732
J recall 0.908 0.077 0.681 0.846 0.608 0.605 0.649 0.724 0.444 0.654 0.447 0.656 0.753 0.737 0.820 0.827 0.830
J decay 0.034 -0.023 0.064 0.059 0.142 0.028 0.030 0.031 0.094 0.073 0.388 0.349 0.171 0.358 0.065 0.241 0.202

F mean 0.672 0.221 0.446 0.640 0.518 0.502 0.536 0.554 0.409 0.554 0.379 0.568 0.608 0.623 0.583 0.701 0.709
F recall 0.791 0.069 0.423 0.750 0.613 0.575 0.622 0.640 0.338 0.620 0.395 0.623 0.780 0.732 0.676 0.837 0.831
F decay 0.063 -0.016 0.064 0.067 0.151 0.024 0.061 0.060 0.091 0.089 0.409 0.336 0.186 0.365 0.072 0.251 0.219
T (GT 0.098) 0.625 0.735 0.744 0.321 0.250 0.305 0.231 0.160 0.578 0.236 0.279 0.130 0.265 0.119 0.278 0.299 0.224

It is not identical to the values in the paper, e.g. mcg J mean is 0.736 when I run it and in the paper it is 0.724.
Why is there a difference?
Also I got different results with the python version as well, I might add more details for that later.

edit: sorry for the bad layout, I'm trying to fix it

[jponttuset]

Hi @pvoigtlaender!
The results in the paper are for the whole set of sequences, whereas the global_table file by default runs it only on the training set. In line 20:

gt_set = 'train';

To get the results on the paper, just change:

gt_set = 'all';

The reason for that is that methods that do training have to be reported only on "val". In our website you can also see the updated tables on the three sets.