π Paper (KDD 2023) β’ π δΈζ README β’ π€ HF Repo [WebGLM-10B] [WebGLM-2B]
This is the official implementation of WebGLM. If you find our open-sourced efforts useful, please π the repo to encourage our following developement!
demo.mp4
Read this in δΈζ.
[2023/06/25] Release ChatGLM2-6B, an updated version of ChatGLM-6B which introduces several new features:
- Stronger Performance: we have fully upgraded the ChatGLM2-6B. It uses the hybrid objective function of GLM, and has undergone pre-training with 1.4T bilingual tokens and human preference alignment training. The evaluation results show that, compared to the first-generation model, ChatGLM2-6B has achieved substantial improvements in performance on datasets like MMLU (+23%), CEval (+33%), GSM8K (+571%), BBH (+60%), showing strong competitiveness among models of the same size.
- Longer Context: Based on FlashAttention technique, we have extended the context length of the base model from 2K in ChatGLM-6B to 32K, and trained with a context length of 8K during the dialogue alignment, allowing for more rounds of dialogue. However, the current version of ChatGLM2-6B has limited understanding of single-round ultra-long documents, which we will focus on optimizing in future iterations.
- More Efficient Inference: Based on Multi-Query Attention technique, ChatGLM2-6B has more efficient inference speed and lower GPU memory usage: under the official implementation, the inference speed has increased by 42% compared to the first generation; under INT4 quantization, the dialogue length supported by 6G GPU memory has increased from 1K to 8K.
More details please refer to ChatGLM2-6Bγ
WebGLM aspires to provide an efficient and cost-effective web-enhanced question-answering system using the 10-billion-parameter General Language Model (GLM). It aims to improve real-world application deployment by integrating web search and retrieval capabilities into the pre-trained language model.
- LLM-augmented Retriever: Enhances the retrieval of relevant web content to better aid in answering questions accurately.
- Bootstrapped Generator: Generates human-like responses to questions, leveraging the power of the GLM to provide refined answers.
- Human Preference-aware Scorer: Estimates the quality of generated responses by prioritizing human preferences, ensuring the system produces useful and engaging content.
- [2023-06-24] We support searching via Bing now!
- [2023-06-14] We release our code and the paper of WebGLM!
Clone this repo, and install python requirements.
pip install -r requirements.txtInstall Nodejs.
apt install nodejs # If you use UbuntuInstall playwright dependencies.
playwright installIf browsing environments are not installed in your host, you need to install them. Do not worry, playwright will give you instructions when you first execute it if so.
In search process, we use SerpAPI to get search results. You need to get a SerpAPI key from here.
Then, set the environment variable SERPAPI_KEY to your key.
export SERPAPI_KEY="YOUR KEY"Alternatively, you can use Bing search with local browser environment (playwright). You can add --searcher bing to start command lines to use Bing search. (See Run as Command Line Interface and Run as Web Service)
Download the checkpoint on Tsinghua Cloud by running the command line below.
You can manually specify the path to save the checkpoint by --save SAVE_PATH.
python download.py retriever-pretrained-checkpointBefore you run the code, make sure that the space of your device is enough.
Export the environment variable WEBGLM_RETRIEVER_CKPT to the path of the retriever checkpoint. If you have downloaded the retriever checkpoint in the default path, you can simply run the command line below.
export WEBGLM_RETRIEVER_CKPT=./download/retriever-pretrained-checkpointYou can try WebGLM-2B model by:
python cli_demo.py -w THUDM/WebGLM-2BOr directly for WebGLM-10B model:
python cli_demo.pyIf you want to use Bing search instead of SerpAPI, you can add --searcher bing to the command line, for example:
python cli_demo.py -w THUDM/WebGLM-2B --searcher bingRun web_demo.py with the same arguments as cli_demo.py to start a web service.
For example, you can try WebGLM-2B model with Bing search by:
python web_demo.py -w THUDM/WebGLM-2B --searcher bingDownload the training data on Tsinghua Cloud by running the command line below.
python download.py generator-training-dataIt will automatically download all the data and preprocess them into the seq2seq form that can be used immediately in ./download.
Please refer to GLM repo for seq2seq training.
Download the training data on Tsinghua Cloud by running the command line below.
python download.py retriever-training-dataRun the following command line to train the retriever. If you have downloaded the retriever training data in the default path, you can simply run the command line below.
python train_retriever.py --train_data_dir ./download/retriever-training-dataYou can reproduce our results on TriviaQA, WebQuestions and NQ Open. Take TriviaQA for example, you can simply run the command line below:
bash scripts/triviaqa.shand start running the experiment.
Here you can see some examples of WebGLM real application scenarios.
When will the COVID-19 disappear?
How to balance career and hobbies?
FL Studio and Cubase, which is better?
Is attention better than CNN?
How to survive in the first-tier cities without a high-salary work?
What do you think of version 3.5 of Genshin Impact?
transformers are originated in NLP, but why they can be applied in CV?
Who proposed Music Transformer? How does it work?
What is the backbone of Toolformer?
If you use this code for your research, please cite our paper.
@misc{liu2023webglm,
title={WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences},
author={Xiao Liu and Hanyu Lai and Hao Yu and Yifan Xu and Aohan Zeng and Zhengxiao Du and Peng Zhang and Yuxiao Dong and Jie Tang},
year={2023},
eprint={2306.07906},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
This repo is simplified for easier deployment.