www-Ye/LLM2GR

Code implementation of the paper [Generative Retrieval with Large Language Models].

Python

Generative Retrieval with Large Language Models

Code implementation of the paper Generative Retrieval with Large Language Models.

Prerequisites

Install or clone the KILT repository.
Install SEAL.

Dataset Download

Download the validation sets of the following datasets from the KILT repository:

NQ
HotpotQA
TriviaQA
ELI5
FEVER
WoW

Follow the instructions in the KILT repository for downloading the specific validation sets.

Generating Retrieval Documents

Step 1: Download the Official KILT Splits

Download kilt_w100_title.tsv and mapping_KILT_title.p.

Step 2: Reconstruct Documents from Segmented Paragraphs

Run the passage2doc.sh script:
```
./passage2doc.sh
```

Step 3: Build Index

Run the build_index.sh script:
```
./build_index.sh
```

Running Generative Retrieval

Execute the run.sh script to perform generative retrieval:
```
./run.sh
```