Trying to run code on IU X-ray database
PurpleDish opened this issue · 4 comments
Hi, I've been interested in image captioning and specifically automatic medical report generation, and I stumbled across your VisualGPT which seemed to take a promising approach, and I've been trying to get it to work with other databases, specifically IU as mentioned in your article.
I can't figure out how you guys have set up the COCO database, and how I should be trying to structure IU X-ray to fit into your code. Is it still supposed to use COCO_detections.hdf5? Or am I supposed to create a hdf5 file for IU?
Hi, I've been interested in image captioning and specifically automatic medical report generation, and I stumbled across your VisualGPT which seemed to take a promising approach, and I've been trying to get it to work with other databases, specifically IU as mentioned in your article.
I can't figure out how you guys have set up the COCO database, and how I should be trying to structure IU X-ray to fit into your code. Is it still supposed to use COCO_detections.hdf5? Or am I supposed to create a hdf5 file for IU?
hi, thanks for interest of our code. for IU X-ray setup, we follow the similar experimental setting as the paper " Generating Radiology Reports via Memory-driven Transformer " in "https://www.aclweb.org/anthology/2020.emnlp-main.112.pdf", where we use ResNet101 pretrained on Imagenet to extract patch features, and feed a sequence of patch features into the visual encoder
Hi, I've been interested in image captioning and specifically automatic medical report generation, and I stumbled across your VisualGPT which seemed to take a promising approach, and I've been trying to get it to work with other databases, specifically IU as mentioned in your article.
I can't figure out how you guys have set up the COCO database, and how I should be trying to structure IU X-ray to fit into your code. Is it still supposed to use COCO_detections.hdf5? Or am I supposed to create a hdf5 file for IU?hi, thanks for interest of our code. for IU X-ray setup, we follow the similar experimental setting as the paper " Generating Radiology Reports via Memory-driven Transformer " in "https://www.aclweb.org/anthology/2020.emnlp-main.112.pdf", where we use ResNet101 pretrained on Imagenet to extract patch features, and feed a sequence of patch features into the visual encoder
Sorry, I'm not sure I understand, so you use ResNet to extract some patch features, and that's the coco_detections.hdf5 file? Also, if you don't mind, how do you create the annotations folder/files
Hi, I've been interested in image captioning and specifically automatic medical report generation, and I stumbled across your VisualGPT which seemed to take a promising approach, and I've been trying to get it to work with other databases, specifically IU as mentioned in your article.
I can't figure out how you guys have set up the COCO database, and how I should be trying to structure IU X-ray to fit into your code. Is it still supposed to use COCO_detections.hdf5? Or am I supposed to create a hdf5 file for IU?hi, thanks for interest of our code. for IU X-ray setup, we follow the similar experimental setting as the paper " Generating Radiology Reports via Memory-driven Transformer " in "https://www.aclweb.org/anthology/2020.emnlp-main.112.pdf", where we use ResNet101 pretrained on Imagenet to extract patch features, and feed a sequence of patch features into the visual encoder
Sorry, I'm not sure I understand, so you use ResNet to extract some patch features, and that's the coco_detections.hdf5 file? Also, if you don't mind, how do you create the annotations folder/files
coco_detections.hdf5 is the precomputed bounding box features of the coco dataset by pretrained Fastter R-CNN. coco_detections.hdf5 is different from IU X-ray features.
you can refer to the paper "Generating Radiology Reports via Memory-driven Transformer " for the IU X-ray feature preparation in more details
Thanks for the help, I'll have a look and see if I can figure it out!