Can not reproduce the reported results.

Question

Can not reproduce the reported results.

blue-blue272 opened this issue a year ago · 3 comments

I load "net_last.pth" for VQ-VAE and "net_best_fid.pth" for the Transformer in 'VQTransformer_corruption05', and run the GPT_eval_multi.py code, and I can only achieve about 0.22 FID, which is higher than the reported 0.116. Can you reproduce the results with the provided weights?

Answer 1 · 2023-11-21T08:47:07.000Z

I encountered similar problem. I used VQ_eval.py to evaluate the 'net_best_fid.pth' provided by the author and found that the FID was 0.278, not 0.070 as in the paper.

Answer 2 · 2024-02-04T10:04:06.000Z

I encountered an issue during HumanML3d data conversion, may I ask how you resolved it?

Answer 3 · 2024-05-12T12:34:43.000Z

I load "net_last.pth" for VQ-VAE and "net_best_fid.pth" for the Transformer in 'VQTransformer_corruption05', and run the GPT_eval_multi.py code, and I can only achieve about 0.22 FID, which is higher than the reported 0.116. Can you reproduce the results with the provided weights?

Can't reproduce the reported metric and got 0.22 FID too TAT. Wondering what's wrong with my testing command. Plz lmk if you have solved this problem. Desperate for help.