/MiniGPT-Med

Open-sourced code of miniGPT-Med

Primary LanguagePython

MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis

Asma Alkhaldi, Raneem Alnajim, Layan Alabdullatef, Rawan Alyahya, Jun Chen, Deyao Zhu, Ahmed Alsinan, Mohamed Elhoseiny

Saudi Data and Artificial Intelligence Authority (SDAIA) and King Abdullah University of Science and Technology (KAUST)

Website link: https://minigpt-med.github.io/

Installation

git clone https://github.com/Vision-CAIR/MiniGPT-Med
cd MiniGPT-Med
conda env create -f environment.yml
conda activate miniGPT-Med

Download miniGPT-Med trained model weights

  • miniGPT-Med's weights miniGPT-Med Model

  • Then modify line 8 at miniGPT-Med/eval_configs/minigptv2_eval.yaml to be the path of miniGPT-Med weight.

Prepare weight for LLMs

Llama2 Version

git clone https://huggingface.co/meta-llama/Llama-2-7b-chat-hf

Then modify line 14 at miniGPT-Med/minigpt4/configs/models/minigpt_v2.yaml to be the path of Llama-2-7b-chat-hf.

Launching Demo Locally

python demo_v2.py --cfg-path eval_configs/minigptv2_eval.yaml --gpu-id 0

Dataset

Dataset Images json file
MIMIC Download Download
NLST Download Downlaod
SLAKE Downlaod Download
RSNA Downlaod Download
Rad-VQA Downalod Download

Acknowledgement

  • MiniGPT-4
  • Lavis
  • Vicuna
  • Falcon
  • Llama 2