Mirror Gradient

By Shanshan Zhong and Zhongzhan Huang and Daifeng Li and Wushao Wen and Jinghui Qin and Liang Lin

This repository is the implementation of "Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima" [paper]. Our paper has been accepted at the 2024 ACM Web Conference (WWW 2024).

🌻 Introduction

Multimodal recommender systems utilize various types of information, such as texts and images, to model user preferences and item features, helping users discover items aligned with their interests. However, it simultaneously magnifies certain risks from multimodal information inputs, such as inherent noise risk and information adjustment risk. To this end, we propose a concise gradient strategy Mirror Gradient (MG) that inverses the gradient signs appropriately during training to make the multimodal recommendation models approach flat local minima easier compared to models with normal training.

🌻 Quick Training

As a general gradient method of recommender systems, MG can be applied to the training of various recommendation models. The code of this repository is based on MMRec, a modern multimodal recommendation toolbox, and the core code of MG is in src/common/trainer.py. We also integrate MG to the modern multimodal recommendation toolbox MMRec

(1) Clone the code.

git clone https://github.com/Qrange-group/Mirror-Gradient
cd Mirror-Gradient

(2) Prepare the enviroment.

conda env create -f environment.yaml
conda activate mg
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
pip install --no-index torch_cluster -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch_sparse -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install --no-index torch_spline_conv -f https://pytorch-geometric.com/whl/torch-2.0.1+cu118.html
pip install torch_geometric

(3) Prepare datasets.

See data. If you want to train models on other Amazon datasets, please see the processing tutorial.

(4) Train the model with MG.

cd src 
python main.py

You can change the model and dataset by the code,

python main.py --model DRAGON --dataset sports
python main.py --model LayerGCN --dataset sports

If you don't want to use MG during training, please use --not_mg. For example,

python main.py --not_mg

(5) Analyze logs.

The training logs are saved in src/logs, and the log file name of models trained with MG has the logo of mg. You can run the following code and get the best result of each log file in src/analysis.txt.

python analysis.py

🌻 Supported Models and Datasets

Models: BM3, DRAGON, DualGNN, FREEDOM, GRCN, ItemKNNCBF, LATTICE, LayerGCN, MGCN, MMGCN, MVGAE, SELFCFED_LGN, SLMRec, VBPR. See models' details in src/models. You also can add others recommendation models to src/models refering to the format of itemknncbf.py.
Datasets: Amazon datasets. You can download the processed datasets baby, sports, clothing, and elec from GoogleDriver. If you want to train models on other Amazon datasets, please see the processing tutorial.

🌻 Checkpoints

For the evaluation of recommendation performance, we pay attention to top-5 accuracy as recommendations in the top positions of rank lists are more important, and adopt four widely used metrics including recall (REC), precision (PREC), mean average precision (MAP), and normalized discounted cumulative gain (NDCG).

You can resume the checkpoint by,

python main.py --model DRAGON --dataset sports --resume DRAGON-sports.pth

Sports

Model	REC	PREC	MAP	NDCG
VBPR	0.0353	0.0079	0.0189	0.0235
VBPR + MG	0.0375	0.0084	0.0203	0.0251
MMGCN	0.0216	0.0049	0.0114	0.0143
MMGCN + MG	0.0241	0.0054	0.0126	0.0158
GRCN	0.0360	0.0080	0.0196	0.0241
GRCN + MG	0.0383	0.0086	0.0207	0.0256
DualGNN	0.0374	0.0084	0.0206	0.0253
DualGNN + MG	0.0387	0.0086	0.0212	0.0261
SLMRec	0.0429	0.0095	0.0233	0.0288
SLMRec + MG	0.0449	0.0099	0.0242	0.0299
BM3	0.0353	0.0078	0.0194	0.0238
BM3 + MG	0.0386	0.0086	0.0210	0.0259
FREEDOM	0.0446	0.0098	0.0232	0.0291
FREEDOM + MG	0.0466	0.0102	0.0242	0.0303
DRAGON	0.0449	0.0098	0.0239	0.0296
DRAGON + MG	0.0465	0.0102	0.0248	0.0307

Elec

Model	REC	PREC	MAP	NDCG
VBPR	0.0182	0.0042	0.0098	0.0122
VBPR + MG	0.0203	0.0046	0.0110	0.0136
MMGCN	0.0140	0.0033	0.0075	0.0094
MMGCN + MG	0.0157	0.0036	0.0084	0.0106
GRCN	0.0226	0.0051	0.0126	0.0155
GRCN + MG	0.0250	0.0057	0.0139	0.0171
DualGNN	0.0238	0.0054	0.0132	0.0162
DualGNN + MG	0.0249	0.0056	0.0139	0.0170
BM3	0.0280	0.0062	0.0157	0.0192
BM3 + MG	0.0285	0.0063	0.0159	0.0195
FREEDOM	0.0252	0.0056	0.0139	0.0171
FREEDOM + MG	0.0260	0.0058	0.0144	0.0176

🌸 Citation

@article{zhong2024mirror,
  title={Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima},
  author={Zhong, Shanshan and Huang, Zhongzhan and Li, Daifeng and Wen, Weushao and Qin, Jinghui and Lin, Liang},
  journal={arXiv preprint arXiv:2402.11262},
  year={2024}
}

💖 Acknowledgments

Many thanks to enoche for their MMRec for multimodal recommendation task.