A problem about the experiments.
Nimo-zl opened this issue · 5 comments
Thank you for sharing the great work! I have a confusion about the experiments:
The running results with this code seem far from the result in the paper. All params are default, after 16 training epochs:
loss=4.445
FDE: 3.1506308334590956
MR(2m,4m,6m): (0.4871615584819174, 0.2329565318727421, 0.13079283688769763)
What could be the problem?
This is result of single trajectory prediction. Performing evaluation will get results of multi-trajectory prediction.
Thank you for your reply! And I have another question:
In evaluate step, it contians "optimize miss rate" and "optimize minFDE", so how can they be optimized after training? And how does the param "MRminFDE" working?
Optimizing strategy during evaluation can be found in the paper. MRminFDE is used to control the ratio of optimizing miss rate. For example, "0.5" means optimizing miss rate and minFDE simultaneously.
Thank you for your reply!
It seems that the results of multi-traj is much better than single-traj, so why is this? During evaluation, how to calculating miss rate and FDE? the average of multi-traj, or the top1? I am so sorry that can't find this in the code.
We use argoverse library to calculate metrics. See https://eval.ai/web/challenges/challenge-page/454/evaluation