🏖️ MEAformer

MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid

This paper introduces MEAformer, a multi-modal entity alignment transformer approach for meta modality hybrid, which dynamically predicts the mutual correlation coefficients among modalities for more fine-grained entity-level modality fusion and alignment.

🔔 News

2023-7 We release the Repo for our paper: Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment !
2023-5 We preprint our paper Revisit and Outstrip Entity Alignment: A Perspective of Generative Models !
2023-4 We release the complete code and data for MEAformer !

🔬 Dependencies

pip install -r requirement.txt

Details

Python (>= 3.7)
PyTorch (>= 1.6.0)
numpy (>= 1.19.2)
Transformers (>= 4.21.3)
easydict (>= 1.10)
unidecode (>= 1.3.6)
tensorboard (>= 2.11.0)

🚀 Train

Quick start: Using script file (run.sh)

>> cd MEAformer
>> bash run.sh

Optional: Using the bash command

>> cd MEAformer
# -----------------------
# ---- non-iterative ----
# -----------------------
# ----  w/o surface  ---- 
# FBDB15K
>> bash run_meaformer.sh 1 FBDB15K norm 0.8 0 
>> bash run_meaformer.sh 1 FBDB15K norm 0.5 0 
>> bash run_meaformer.sh 1 FBDB15K norm 0.2 0 
# FBYG15K
>> bash run_meaformer.sh 1 FBYG15K norm 0.8 0 
>> bash run_meaformer.sh 1 FBYG15K norm 0.5 0 
>> bash run_meaformer.sh 1 FBYG15K norm 0.2 0 
# DBP15K
>> bash run_meaformer.sh 1 DBP15K zh_en 0.3 0 
>> bash run_meaformer.sh 1 DBP15K ja_en 0.3 0 
>> bash run_meaformer.sh 1 DBP15K fr_en 0.3 0
# ----  w/ surface  ---- 
# DBP15K
>> bash run_meaformer.sh 1 DBP15K zh_en 0.3 1 
>> bash run_meaformer.sh 1 DBP15K ja_en 0.3 1 
>> bash run_meaformer.sh 1 DBP15K fr_en 0.3 1
# -----------------------
# ------ iterative ------
# -----------------------
# ----  w/o surface  ---- 
# FBDB15K
>> bash run_meaformer_il.sh 1 FBDB15K norm 0.8 0 
>> bash run_meaformer_il.sh 1 FBDB15K norm 0.5 0 
>> bash run_meaformer_il.sh 1 FBDB15K norm 0.2 0 
# FBYG15K
>> bash run_meaformer_il.sh 1 FBYG15K norm 0.8 0 
>> bash run_meaformer_il.sh 1 FBYG15K norm 0.5 0 
>> bash run_meaformer_il.sh 1 FBYG15K norm 0.2 0 
# DBP15K
>> bash run_meaformer_il.sh 1 DBP15K zh_en 0.3 0 
>> bash run_meaformer_il.sh 1 DBP15K ja_en 0.3 0 
>> bash run_meaformer_il.sh 1 DBP15K fr_en 0.3 0
# ----  w/ surface  ---- 
# DBP15K
>> bash run_meaformer_il.sh 1 DBP15K zh_en 0.3 1 
>> bash run_meaformer_il.sh 1 DBP15K ja_en 0.3 1 
>> bash run_meaformer_il.sh 1 DBP15K fr_en 0.3 1

❗Tips: you can open the run_meaformer.sh or run_meaformer_il.sh file for parameter or training target modification.

🎯 Results

$\bf{H@1}$ Performance with the Settings: w/o surface & Non-iterative in UMAEA. We modified part of the MSNEA to involve not using the content of attribute values but only the attribute types themselves (See issues for details):

Method	$\bf{DBP15K_{ZH-EN}}$	$\bf{DBP15K_{JA-EN}}$	$\bf{DBP15K_{FR-EN}}$
MSNEA	.609	.541	.557
EVA	.683	.669	.686
MCLEA	.726	.719	.719
MEAformer	.772	.764	.771
UMAEA	.800	.801	.818

📚 Dataset

❗NOTE: Download from GoogleDrive (1.26G) and unzip it to make those files satisfy the following file hierarchy:

ROOT
├── data
│   └── mmkg
└── code
    └── MEAformer

Code Path

👈 🔎 Click

MEAformer
├── config.py
├── main.py
├── requirement.txt
├── run_meaformer.sh
├── run_meaformer_il.sh
├── run.sh
├── model
│   ├── __init__.py
│   ├── layers.py
│   ├── MEAformer_loss.py
│   ├── MEAformer.py
│   ├── MEAformer_tools.py
│   └── Tool_model.py
├── src
│   ├── __init__.py
│   ├── distributed_utils.py
│   ├── data.py
│   └── utils.py
└── torchlight
    ├── __init__.py
    ├── logger.py
    ├── metric.py
    └── utils.py

Data Path