Presenting initial findings with ongoing work. Updates will follow as progress continues.
The integration of multi-modal data has emerged as a promising approach in various fields, enabling a more comprehensive understanding of complex phenomena by leveraging the complementary information from different sources. In the realm of medical research, the integration of multi-modal data has emerged as a powerful approach for enhancing our understanding of complex diseases and conditions. The fusion of different data types, such as tabular data (electronic health records - EHR) encompassing medical records and demographic information, together with high-resolution imaging modalities like MRI scans, has unlocked new avenues for comprehensive analysis and diagnosis.
In this work, we propose a novel approach that harnesses the power of hypernetworks to fuse tabular data and MRI brain scans.
Training a network,
We use the tabular information as an input to the Hypernetwork (
We demonstrate the versatility and efficacy of the proposed hypernetwork framework, named HyperFusion, through two distinct brain MRI analysis tasks: brain age prediction conditioned by the subject's sex and classification of subjects into Alzheimer's disease (AD), Mild Cognitive Impairment (MCI), and Cognitively Normal (CN) groups conditioned by their tabular data, which includes clinical measurements, as well as demographic and genetic information.
The ADNI dataset, ADNI 1, ADNI 2 and ADNI GO, baseline visits
ADNI aims to standardize the data collection methods and promote the use of it for research to accelerate discoveries in the disease
2120 MRI scans - healthy, MCI and AD patients (34%, 48%, 17%)
5 folds (~420 samples each)– one for testing and 4 for cross-validation (same distribution of labels)
The Tabular features used (9):
- demographic: Age, Sex, Education (years)
- genetic risk factor: ApoE4
- cerebrospinal fluid biomarkers : Abeta42, P-tau181, T-tau
- measures derived from PET scan: 18 F-fluorodeoxyglucose (FDG) florbetapir (AV)
