Doubt on quantitative result table.

[CHELSEA234]

Hi,

Firstly, I want to congratulate you for this paper being cited for multiple times recently.

However, I still feel confused about some details. I double check your paper where you successfully reproduced previous works (SRNN) in CVPR2016 and published results of them:

Is it the result that you optimized these models over 400ms or 1000ms? I guess that result above is one being optimized over 400ms, the one that I traced through here github with source code is optimized over 1000ms, which means model predicts for long term. I also attached your posted result here:

Can you help out here?

Best regards,
CHELSEA234

[una-dinosauria]

Hi @CHELSEA234,

As you can see in the issue you linked, I took the pretrained models that the authors provided.
Right now I do not remember the details of their implementation (and I didn't have to know to plot that table, because I was only evaluating, not training).
You may have to dig into their code and paper to find out.

Cheers,

[CHELSEA234]

@una-dinosauria Thanks for your reply, it is my fault that I didn't dig into their source code before posting this issue.

I just read your history comments in SRNN, I realized that what you did was evaluating their pretrained model and recording the results as follow:

1. merr[2,4,8,10] (corresponding to [80, 160, 320, 400]/ figure 1 in my post) for your published paper.
2. merr[2,4,8,14,25] (corresponding to [80, 160, 320, 560 and 1000]/ figure 2 in my post) for your comments of issue I linked.

This also explains the difference in your paper's results and SRNN's, I appreciate your great efforts here. BTW, any chance could you share your complete results merr[1,2...1000] in public, if you have. I would be more grateful to your help.

Best,
CHELSEA234

==============================
@una-dinosauria Can you give some suggestions here?

[una-dinosauria]

Hi @CHELSEA234,

I'm sorry for the late reply.

BTW, any chance could you share your complete results merr[1,2...1000] in public, if you have. I would be more grateful to your help.

Sorry, I don't have the results handy. Is this something that can't be reproduced from the provided code? Training the models yourself would probably be the fastest way to recover the complete merr (mean squared error?) curves. I don't remember being particularly expensive to train (perhaps a day or two to get all the results with a GTX-1080 GPU).

Cheers,