MedKLIP:Medical Knowledge Enhanced Language-Image Pre-Training

Introduction:

The official implementation code for "MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training".

Check checkpoints dir to download our pre-trained model. It can be used for all zero-shot && finetuning tasks

Zero-Shot Classification:

We give an example on CXR14 in Sample_Zero-Shot_Classification_CXR14. Modify the path, and test our model by python test.py
Zero-Shot Grounding:

We give an example on RSNA_Pneumonia in Sample_Zero-Shot_Grounding_RSNA. Modify the path, and test our model by python test.py
Finetuning:

We give segmentation and classification finetune code on SIIM_ACR dataset in Sample_Finetuning_SIIMACR. Modify the path, and finetune our model by python I1_classification/train_res_ft.py or python I2_segementation/train_res_ft.py

Our pre-train code is given in Train_MedKLIP.

Check the Train_MedKLIP\data_file dir and download the pre-processed data files.
Modify the path as you disire, and python PreTrain_MedKLIP\train_MedKLIP.py to pre-train.

We borrow some pre-process codes from AGXnet

If you have any question, please feel free to contact wtzxxxwcy02@sjtu.edu.cn.