ChatGPT-Benchmark-on-Radiation-Oncology
Benchmarking ChatGPT-4 on a radiation oncology in-training exam and Red Journal Gray Zone cases: potentials and challenges for AI-assisted medical education and decision making in radiation oncology
ACR TXIT Exam
This repository contains the American College of Radiation (ACR) radiation oncology in-training (TXIT) exam with annotations whether ChatGPT-3.5 and ChatGPT-4 answer the questions incorrectly (red 0: false answer by ChatGPT-3.5; blue 0: false answer by ChatGPT-4). The Excel file contails all the reponses from both versions of ChatGPT. Each sheet contains 30 questions. In the last two sheets, the distributions of correct answers based on certain topics are analysed.
The exam sheet and the knowledge domain definition are from the ACR official website.
Red Journal Gray Zone Cases
This repository contains the 2022 red journal collection of gray zone cases as well as ChatGPT-4's responses. For each case, the case description and the corresponding human experts' opinions are publicly available via the red journal gray zone website. ChatGPT-4's recommendations for treating the patient are attached at the end of the corresponding pdf file for each case. In addition, the files we used for the blind clinician vote are also displayed.