How to obtain the mtrics of PAB

Question

How to obtain the mtrics of PAB

LiewFeng opened this issue 4 months ago · 6 comments

PAB is a really useful work and I wonder how obtain the mtrics of PAB. Thanks a lot!

Answer 1 · 2024-08-24T05:17:05.000Z

do you mean the evaluation results? it's in the eval dir

Answer 2 · 2024-08-26T02:13:15.000Z

Yes. I notice the files in the eval dir but there is no instruction file and could you provide more detailed instructions? Thanks!

Answer 3 · 2024-08-26T08:40:36.000Z

sure will soon

Answer 4 · 2024-09-03T21:06:21.000Z

guide for eval is updated now: https://github.com/NUS-HPC-AI-Lab/VideoSys/blob/master/eval/pab/README.md

Answer 5 · 2024-09-06T02:06:57.000Z

Thanks! Shoud 'enable_flash_attn' be True by default? Since 'We enable FlashAttention (Dao et al., 2022) by default for all experiments.' is claimed in the paper.

Answer 6 · 2024-09-07T17:44:21.000Z

currently torch already support flash attention. so we dont require to install flashattn manually.