lupantech/chameleon-llm

Questions on TabWMP

Jungle728 opened this issue · 1 comments

Thank you for your work. When running the TabWMP dataset, I found that some of the examples are executing very slowly, is there any way you can speed them up?
It only takes 4-5 hours to complete on the Science QA dataset, but it can take 20 times longer on the TabWMP dataset.
Looking forward to hearing from you, thanks!

Thank you for your feedback. Generally, the duration of downstream tasks primarily depends on the number of test examples. In our experiments, the processing times for Science and TabWMP were relatively similar.

Could you please provide more details, such as the running log or the code snippet that leads to the time bottleneck, to better identify the cause? Thank you!