ViperX7/Alpaca-Turbo

No model is shown in web-ui with docker

Cabanera opened this issue · 9 comments

Cabanera commented

OS: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
Docker version:

Client:
 Version:           20.10.5+dfsg1
 API version:       1.41
 Go version:        go1.15.15
 Git commit:        55c4c88
 Built:             Mon May 30 18:34:49 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.5+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.15
  Git commit:       363e9a8
  Built:            Mon May 30 18:34:49 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.13~ds1
  GitCommit:        1.4.13~ds1-1~deb11u3
 runc:
  Version:          1.0.0~rc93+ds1
  GitCommit:        1.0.0~rc93+ds1-5+deb11u2
 docker-init:
  Version:          0.19.0
  GitCommit:

I've followed the installation guide for docker, and tried adding the following models in the models directory:

But in the webui no model is shown
Image webui

No error log with docker logs

ViperX7 commented

what command you used to run alpaca turbo using Docker

Cabanera commented

docker-compose up

ViperX7 commented

send me the output of

can you get inside the container using
docker-compose exec alpaca /bin/bash

and show me the output of
ls /app/models/

Cabanera commented

send me the output of

can you get inside the container using docker-compose exec alpaca /bin/bash

and show me the output of ls /app/models/

ls /app/models/
ggml-alpaca-7b-q4.bin ggml-model-q4_0.bin ggml-model-q4_1.bin

ViperX7 commented

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

Cabanera commented

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :

[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[
Cabanera commented

With another model

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :

[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[

with another model the previous error disappears but in webui it remains stuck in loading. And the log fle is the following:

[
'bin/main',
'-i',
'--seed',
 '888777',
'-ins',
'-t',
 '4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
'--repeat_last_n',
'64',
'--repeat_penalty',
'1',
'--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]

Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

That "Working" complete in 0 sec seems suspicious.

  • 👍1
samex commented

Same issue, the model sits inside docker container but the webui is not showing it

Spiider2020 commented

I had the same issue.
For me the problem was I was running Docker on my server (network machine).
The way I got it working is by getting the source code and editing it and then rebuilding.

Load the ui folder into your code editor.
Go to src/app/environments/environment.prod.ts and change the https site to http://(network machine ip):5000

Then go to angular.json find and change the config as bellow:

"configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "src/app/environments/environment.ts",
                  "with": "src/app/environments/environment.prod.ts"
                }
              ],
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "500kb",
                  "maximumError": "1mb"
                },
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "2kb",
                  "maximumError": "4kb"
                }
              ],
              "outputHashing": "all"
            },

Then

npm install
npm run build

Go into your docker folder where you have your project (alpaca-turbo folder), into templates , create a folder called whatever you like and move the contents of templates to it.
Then copy everything from /dist/angular-turbo where you built the angular ui to alpaca-turbo/templates .

Now you should see your .bin files

Edit:
Or even easier , you can do what this guy said
#39 (comment)