Query about the VisEvent results and reproduction problem

[LingyiHongfd]

Thanks for your interesting work. When I run the training experiment following the same setting as the parameters you provided, I find it is hard to reproduce the results in paper. I only achieves about 0.56 F-score in DepthTrack (0.594 in your paper) and about 0.64 on LasHeR (0.651 in your paper). I can not reproduce the results reported in your paper with several training. Can you provide more details on the training settings?
Besides, I evaluate the raw results of VisEvent which is provided by you using the official matlab codes, I only obtain 0.731 and 0.592m while the results in your paper is 0.758 and 0.731. Can you give me more information about ViPT on VisEvent?
If you can reply my queries, I would appreciate it very much.

[jiawen-zhu]

Are you using the same parameter settings such as batchsize when you reproduce it? We have also reproduced the results on other devices, for example, on DethTrack, the reproduced results can easily achieve the performance in the paper. In fact, our experimental settings are all set up in the codes.

For VisEvent, the results in our paper are 0.758 and 0.592 for precision and success scores, not "0.758 and 0.731" as you mentioned. In addition, I just downloaded the raw results again and evaluated it, and the results are consistent with the paper.

I suggest you confirm that the parameter settings, training set, testing and evaluation process are correct and follow the setting in the paper.

[laisimiao]

@LingyiHongfd Due to the historical reasons of the VISEVENT dataset, there may be many minor issues in it. Therefore, we recommend that you use our organized toolkit: https://drive.google.com/file/d/1EY3H6YzsCEE2Tss5z3ROaWaXT6icZVui/view?usp=drive_link