Vision-CAIR/VisualGPT

Running code on IU X-Ray dataset

dwij2212 opened this issue · 2 comments

dwij2212 commented

Hello, I am interested in running your Visual GPT model on the IU Xray dataset. Can you please explain how I can use this model to train on the dataset? I saw issue #4 but I was not able to understand how to create a .h5 file for the IU-Xray dataset.
Could you please walk me through how I can set up the .h5 file for IU-Xray?

junchen14 commented

Hi, we directly use the features that are provided in the paper " Generating Radiology Reports via Memory-driven Transformer " in "https://www.aclweb.org/anthology/2020.emnlp-main.112.pdf".

You can either create the same .h5 file as the one we provided for COCO dataset, or you can modify the dataloader to fit their data format.

KaratzasBasil commented

Is it easy to tell me the hyperparameters for the model? I managed to get it to work on IU X-Ray but my first runs have Rouge below 30 and I am wondering if it has to do with the hyperparameters or something else from the code I wrote to make it work on IU X-Ray.

Thanks :)