Lost connection between UI and server

[benoit-cty]

Expected Behavior

When asking something to the model, the answer appears in the UI.

Current Behavior

The terminal show that he answered but nothing appear in the UI.

Steps to Reproduce

1. Use Falcon-40B
2. Ask something
3. Wait until "Done" show in terminal
4. Nothing happen
5. Reload the UI
6. The answers appears in the UI. Strange !

Possible Solution

Keep WebSocket open for a longer time ?
I try to change ping_timeout of SocketIO but without success.

Context

Terminal trace:

Text generation requested by client: GH1KDxn5pCTcpVtEAAAD
Received message : Quel est le sens de la vie ?
warmup

Client BlaDeHD1jPsQJCq9AACA disconnected
Client 5-Ce4DsY9IpryyxxAACC connected

Client 5-Ce4DsY9IpryyxxAACC disconnected
Client YJ3dp2GHdzNIKVZpAACE connected
generated:152 words
Client YJ3dp2GHdzNIKVZpAACE disconnected
Finished executing the generation

## Done Generation ##


## Done ##

Same problem while downloading model from the UI : the progress bar stop being updated in the UI but downloads continues in the terminal.

EDIT : With orca-mini-3b.ggmlv3.q4_0.bin it works.

[ParisNeo]

I guess it is due to the size of 40B it is enormous et can't be fully handled by gpu, so many things use cpu, and i guess this makes the backend non responsive.
I have changed the backend to use eventlet, this allows multi connections that will hopefully reduce disconnections.