deep-diver/LLM-As-Chatbot

Something went wrong Connection errored out. Loading checkpoint shards

javierxio opened this issue · 4 comments

javierxio commented

After going to the url and selecting the model, after download, it breaks at checkpoint.

To create a public link, set `share=True` in `launch()`.
determined model type: falcon
Downloading (…)okenizer_config.json: 100%|█████| 220/220 [00:00<00:00, 25.0kB/s]
Downloading (…)/main/tokenizer.json: 100%|█| 2.73M/2.73M [00:00<00:00, 19.2MB/s]
Downloading (…)cial_tokens_map.json: 100%|██████| 281/281 [00:00<00:00, 230kB/s]
cpu mode
Downloading (…)lve/main/config.json: 100%|██████| 667/667 [00:00<00:00, 436kB/s]
Downloading (…)/configuration_RW.py: 100%|█| 2.61k/2.61k [00:00<00:00, 1.86MB/s]
A new version of the following files was downloaded from https://huggingface.co/tiiuae/falcon-7b-instruct:
- configuration_RW.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Downloading (…)main/modelling_RW.py: 100%|█| 47.5k/47.5k [00:00<00:00, 34.7MB/s]
A new version of the following files was downloaded from https://huggingface.co/tiiuae/falcon-7b-instruct:
- modelling_RW.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Downloading (…)model.bin.index.json: 100%|█| 16.9k/16.9k [00:00<00:00, 11.9MB/s]
Downloading (…)l-00001-of-00002.bin: 100%|█| 9.95G/9.95G [02:53<00:00, 57.3MB/s]
Downloading (…)l-00002-of-00002.bin: 100%|█| 4.48G/4.48G [01:18<00:00, 57.4MB/s]
Downloading shards: 100%|████████████████████████| 2/2 [04:12<00:00, 126.13s/it]
Loading checkpoint shards:  50%|█████████         | 1/2 [00:04<00:04,  4.98s/it]Killed

Screenshot from 2023-06-10 20-01-10

javierxio commented

I see ubuntu logs and see this message when the checkpoint is Killed

Out of memory: Killed process 23122 (python) total-vm:16368736kB, anon-rss:12446840kB, file-rss:0kB, shmem-rss:4kB, UID:1000 pgtables:25712kB oom_score_adj:0
deep-diver commented

can you share your system info?

javierxio commented

I am in cpu mode.

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS

     *-cpu
          description: CPU
          product: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
          vendor: Intel Corp.

I changed the swappiness from 60 (default) to 80 and it seems to help me get past that error.

cat /proc/sys/vm/swappiness
Output
60

To change swappiness:

sudo sysctl vm.swappiness=80

More info here: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-20-04

i am running it on the browser but it is super slow. maybe one word every couple of minutes. is this expected?

deep-diver commented
  • 👍1