Reading List for Multimodal Learning in Healthcare

Papers

Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I. & Lungren, M. P. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. npj Digit. Med. 3, 136 (2020). html
Behrad, F. & Saniee Abadeh, M. An overview of deep learning methods for multimodal medical data mining. Expert Systems with Applications 200, 117006 (2022). html

Khader, F. et al. Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers. Radiology 309, e230806 (2023).html
Barros, V. et al. Virtual Biopsy by Using Artificial Intelligence-based Multimodal Modeling of Binational Mammography Data. Radiology 220027 (2022) pdf
Qiu, S. et al. Multimodal deep learning for Alzheimer’s disease dementia assessment. Nature Communications 17 (2022). pdf
Xu, M. et al. Accurately Differentiating Between Patients With COVID-19, Patients With Other Viral Infections, and Healthy Individuals: Multimodal Late Fusion Learning Approach. J Med Internet Res 23, e25535 (2021). pdf
Mei, X. et al. Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat Med 26, 1224–1228 (2020).html

Silva, J. F. & Matos, S. Modelling patient trajectories using multimodal information. J Biomed Inform 134, 104195 (2022).html
Liu, S. et al. Multimodal Data Matters: Language Model Pre-Training Over Structured and Unstructured Electronic Health Records. IEEE J Biomed Health Inform PP, (2022).pdf
Darabi, S., Kachuee, M., Fazeli, S. & Sarrafzadeh, M. TAPER: Time-aware patient EHR representation. IEEE Journal of Biomedical and Health Informatics 24, 3268–3275 (2020).pdf

Sundrani, S. et al. Predicting patient decompensation from continuous physiologic monitoring in the emergency department. Npj Digit Med 6, 1–10 (2023).html
Kim, H. B. et al. Computational signatures for post-cardiac arrest trajectory prediction: Importance of early physiological time series. Anaesth Crit Care Pa 41, 101015 (2022).html
Chen, H., Lundberg, S. M., Erion, G., Kim, J. H. & Lee, S.-I. Forecasting adverse surgical events using self-supervised transfer learning for physiological signals. npj Digit. Med. 4, 1–13 (2021).html
Feng, Y. et al. DCMN: Double Core Memory Network for Patient Outcome Prediction with Multimodal Data. IEEE International Conference on Data Mining (ICDM) 10 (2019).html
Xu, Y., Biswal, S., Deshpande, S. R., Maher, K. O. & Sun, J. RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data. in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2565–2573 (ACM, 2018).html

Soenksen, L. R. et al. Integrated multimodal artificial intelligence framework for healthcare applications. npj Digit. Med. 5, 149 (2022).html
Boehm, K. M. Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer. Nature Cancer 3, 24 (2022).html
Golovanevsky, M., Eickhoff, C. & Singh, R. Multimodal attention-based deep learning for Alzheimer’s disease diagnosis. J Am Med Inform Assn ocac168 (2022) pdf