Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)

Paper link: https://arxiv.org/pdf/2203.13366.pdf

Introduction

We present a flexible and unified text-to-text paradigm called "Pretrain, Personalized Prompt, and Predict Paradigm'' (P5) for recommendation, which unifies various recommendation tasks in a shared framework. In P5, all data such as user-item interactions, item metadata, and user reviews are converted to a common format -- natural language sequences. Specifically, P5 learns different tasks with the same language modeling objective during pretraining. Thus, it possesses the potential to serve as the foundation model for downstream recommendation tasks, allows easy integration with other modalities, and enables instruction-based recommendation, which will revolutionize the technical form of recommender system towards universal recommendation engine. With adaptive personalized prompt for different users, P5 is able to make predictions in a zero-shot or few-shot manner and largely reduces the necessity for extensive fine-tuning. On several recommendation benchmarks, we conduct experiments to show the effectiveness of our generative approach.

Requirements:

Python 3.9.7
PyTorch 1.10.1
transformers 4.2.1
tqdm
numpy
sentencepiece
pyyaml

Usage

Clone this repo

git clone https://github.com/jeykigung/P5.git

Download preprocessed data from this Google Drive link, then put them into the data folder. If you would like to preprocess your own data, please follow the jupyter notebooks in the preprocess folder. Raw data can be downloaded from this Google Drive link, then put them into the raw_data folder.
Download pretrained checkpoints into snap folder. If you would like to train your own P5 models, snap folder will also be used to store P5 checkpoints.
Pretrain with scripts in scripts folder, such as
```
bash scripts/pretrain_P5_base_beauty.sh 4
```
Evaluate with example jupyter notebooks in the notebooks folder. Before testing, create a soft link of data folder to the notebooks folder by
```
cd notebooks
ln -s ../data .
```

Pretrained Checkpoints

See CHECKPOINTS.md.

Citation

Please cite the following paper corresponding to the repository:

@inproceedings{geng2022recommendation,
  title={Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt \& Predict Paradigm (P5)},
  author={Geng, Shijie and Liu, Shuchang and Fu, Zuohui and Ge, Yingqiang and Zhang, Yongfeng},
  booktitle={Sixteenth ACM Conference on Recommender Systems},
  year={2022}
}

Acknowledgements

VL-T5, PETER, and S3-Rec

evison/P5