This repository is the PyTorch impelementation for the paper "Federated Recommendation via Hybrid Retrieval Augmented Generation".
We propose GPT-FedRec, a federated recommendation framework leveraging ChatGPT and a novel hybrid Retrieval Augmented Generation (RAG) mechanism. GPT-FedRec is a two-stage solution. The first stage is a hybrid retrieval process, mining ID-based user patterns and text-based item features. Next, the retrieved results are converted into text prompts and fed into GPT for re-ranking. Our proposed hybrid retrieval mechanism and LLM-based re-rank aims to extract generalized features from data and exploit pretrained knowledge within LLM, overcoming data sparsity and heterogeneity in FR. In addition, the RAG approach also prevents LLM hallucination, improving the recommendation performance for real-world users.
For our running environment see requirements.txt
- For Beauty, Games, Auto and ML-100K, they will be automatically downloaded and processed, when the training scripts are executed.
- For Toys, please go to this repo
- download the data, and place
datamaps.json
,meta.json.gz
,sequential_data.txt
into thetoys_new
folder - we call it
toys_new
, because it is already processed.
- download the data, and place
- Example folder structure
├── ...
├── data
│ ├── auto
│ ├── beauty
│ ├── toys_new
│ └── ...
├── dataloader
│ └── ...
├── datasets
│ └── ...
└── ...
-
Stage 1: train LRURec and E5 (Hybrid Retrieval)
- Example
python train_lru.py --dataset_code --num_clients --num_samples --global_epochs --local_epochs --lr python train_e5.py --dataset_code --num_clients --num_samples --global_epochs --local_epochs --lr
- Hyperparameters
--dataset_code # select from 'beauty', 'games', 'toys_new', 'auto', 'ml-100k' --num_clients # number of federated clients --num_samples # number of training users per client --global_epochs # global epochs for federated learning --local_epochs # local epochs for federated learning --lr # learning rate
- After executing the script, the test scores will be automatically saved to a new folder
experiments
- Example
-
Stage 2: Perform re-rank with GPT-3.5 (RAG-based Re-ranking)
- Example
python fuse.py --dataset_code --openai_token --num_clients --num_samples --lambda_ensemble
- Hyperparameters
--openai_token # the token to call the OpenAI API --dataset_code # select from 'beauty', 'games', 'toys_new', 'auto', 'ml-100k' --num_clients # number of federated clients --num_samples # number of training users per client --lambda_ensemble # the trade-off factor when computing the hybrid retrival score (Equation 6 in our paper)
- After executing the script, the generated texts will be saved to a new folder
llm_results
.
- Example
-
Evaluation:
- Example
python eval_fused.py --dataset_code --num_clients --num_samples --lambda_ensemble
- After executing the script, the Recall and NDCG scores will be printed.
- Example
The table below reports our main performance results, with best results marked in bold and second best results underlined. For training and evaluation details, please refer to our paper.
During the implementation, we base our code mostly on existing repos (e.g., LRURec). Many thanks to these authors for their great work!