Self host v1.4.0 MODEL always /infengine-v1/completions-wait-batch WAIT time out

Question

Self host v1.4.0 MODEL always /infengine-v1/completions-wait-batch WAIT time out

Closed this issue a year ago · 10 comments

After upgrade the docker image to v1.4.0(latest), my vscode refact can't get an inference from myself host server
and I check the logs it always show :


-- 68 -- 20240214 07:58:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:58:40 MODEL 10004.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:58:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:58:44 MODEL 10005.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:58:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:58:50 MODEL 10004.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:58:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:58:54 MODEL 10004.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:00 MODEL 10005.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:59:04 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:10 MODEL 10005.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:59:14 MODEL 10005.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:20 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:59:24 MODEL 10004.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:30 MODEL 10004.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 07:59:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 1389 -- 20240214 07:59:34 MODEL 10004.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:40 MODEL 10003.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:59:44 MODEL 10003.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 07:59:50 MODEL 10004.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 07:59:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 07:59:54 MODEL 10004.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:00 MODEL 10002.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:00:04 MODEL 10005.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:10 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:00:14 MODEL 10017.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:20 MODEL 10008.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:00:24 MODEL 10012.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:30 MODEL 10013.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 08:00:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 1389 -- 20240214 08:00:34 MODEL 10006.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:40 MODEL 10017.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:00:44 MODEL 10007.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:00:50 MODEL 10017.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:00:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:00:54 MODEL 10005.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:00 MODEL 10010.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:01:04 MODEL 10005.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:10 MODEL 10009.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:01:14 MODEL 10007.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:20 MODEL 10005.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:01:24 MODEL 10012.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:30 MODEL 10010.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 08:01:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 1389 -- 20240214 08:01:34 MODEL 10011.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:40 MODEL 10015.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:01:44 MODEL 10004.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:01:50 MODEL 10013.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:01:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:01:54 MODEL 10005.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:00 MODEL 10012.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:02:04 MODEL 10007.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:10 MODEL 10013.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:02:14 MODEL 10005.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:20 MODEL 10012.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:02:24 MODEL 10009.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:30 MODEL 10013.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 08:02:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 1389 -- 20240214 08:02:34 MODEL 10013.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:40 MODEL 10007.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:02:44 MODEL 10009.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:02:50 MODEL 10011.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:02:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:02:54 MODEL 10011.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:00 MODEL 10006.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:03:04 MODEL 10009.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:10 MODEL 10007.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:03:14 MODEL 10007.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:20 MODEL 10005.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:03:24 MODEL 10005.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:30 MODEL 10012.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 08:03:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 1389 -- 20240214 08:03:34 MODEL 10007.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:40 MODEL 10011.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:03:44 MODEL 10009.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:03:50 MODEL 10006.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:03:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:03:54 MODEL 10009.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:04:00 MODEL 10008.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:04 MODEL 10009.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:04:10 MODEL 10009.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:14 MODEL 10008.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:04:20 MODEL 10006.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:24 MODEL 10010.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:04:30 MODEL 10006.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:34 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:34 MODEL 10008.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:04:40 MODEL 10010.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:44 MODEL 10004.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:48 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:48 WEBUI 192.168.250.12:56460 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:56460 - "GET / HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:56460 - "GET /style.css HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /index.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /error.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /list-plugins HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /tab-model-hosting.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:56460 - "GET /favicon.png HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET / HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:56460 - "GET /tab-upload.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57470 - "GET /tab-model-hosting.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:56460 - "GET /components/modals/modal-upload-files.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57470 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /tab-host-models-get HTTP/1.1" 200
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57464 - "GET /tab-finetune.js HTTP/1.1" 200
-- 1263 -- 20240214 08:04:50 MODEL 10009.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:50 WEBUI 192.168.250.12:57470 - "GET /tab-upload.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /tab-server-logs.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-finetune.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /tab-stats.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:56460 - "GET /tab-server-logs.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-finetune-progress-svg/none HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /dashboards/dash_prime.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:56460 - "GET /dashboards/dash_users.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /dashboards/utils.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /dashboards/plots.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-settings.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /tab-stats.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-about.js HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /tab-settings.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-about.html HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57470 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:51 WEBUI 192.168.250.12:57464 - "GET /tab-host-models-get HTTP/1.1" 200
-- 68 -- 20240214 08:04:53 WEBUI 192.168.250.12:57464 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:04:54 MODEL 10005.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:04:55 WEBUI 192.168.250.12:57464 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:57 WEBUI 192.168.250.12:57464 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:59 WEBUI 192.168.250.12:57464 - "GET /tab-host-have-gpus HTTP/1.1" 200
-- 68 -- 20240214 08:04:59 WEBUI 192.168.250.12:57464 - "GET /tab-server-log-get HTTP/1.1" 200
-- 68 -- 20240214 08:04:59 WEBUI 192.168.250.12:57470 - "GET /tab-server-log-plain/latest?stream=1 HTTP/1.1" 200
-- 68 -- 20240214 08:05:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:00 MODEL 10005.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:04 MODEL 10005.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:10 MODEL 10006.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:14 MODEL 10005.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:20 WEBUI 192.168.250.12:57464 - "GET /tab-server-log-plain/watchdog_20240214.log HTTP/1.1" 200
-- 68 -- 20240214 08:05:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:20 MODEL 10007.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:21 WEBUI 192.168.250.12:57464 - "GET /favicon.ico HTTP/1.1" 404
-- 68 -- 20240214 08:05:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:24 MODEL 10005.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:30 MODEL 10010.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:34 MODEL 10005.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:40 MODEL 10004.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:44 MODEL 10005.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:05:50 MODEL 10005.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:05:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:05:54 MODEL 10005.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:00 MODEL 10004.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:04 MODEL 10005.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:10 MODEL 10008.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:14 MODEL 10005.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:20 MODEL 10007.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:24 MODEL 10005.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:30 MODEL 10008.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:34 MODEL 10006.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:40 MODEL 10010.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:44 MODEL 10005.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:06:50 MODEL 10005.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:06:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:06:54 MODEL 10004.5ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:00 MODEL 10007.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:04 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:10 MODEL 10007.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:14 MODEL 10008.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:20 MODEL 10006.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:24 MODEL 10008.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:30 MODEL 10008.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:34 MODEL 10005.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:40 MODEL 10004.7ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:44 MODEL 10005.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:07:50 MODEL 10004.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:07:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:07:54 MODEL 10003.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:00 MODEL 10004.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:04 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:10 MODEL 10003.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:14 MODEL 10005.2ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:20 MODEL 10004.3ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:24 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:24 MODEL 10003.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:30 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:30 MODEL 10004.6ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:34 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:34 MODEL 10004.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:40 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:40 MODEL 10004.9ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:44 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:44 MODEL 10004.1ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:50 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:08:50 MODEL 10004.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:08:54 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:08:54 MODEL 10002.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:09:00 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:09:00 MODEL 10003.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:09:04 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:09:04 MODEL 10004.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:09:10 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:09:10 MODEL 10004.8ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:09:14 WEBUI 127.0.0.1:55430 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1389 -- 20240214 08:09:14 MODEL 10005.0ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT
-- 68 -- 20240214 08:09:20 WEBUI 127.0.0.1:55418 - "POST /infengine-v1/completions-wait-batch HTTP/1.1" 200
-- 1263 -- 20240214 08:09:20 MODEL 10005.4ms http://127.0.0.1:8008/infengine-v1/completions-wait-batch WAIT

any tips? pls

Answer 1 · 2024-02-14T08:12:20.000Z

here is the full logs:
watchdog_20240214.log

Answer 2 · 2024-02-14T14:43:43.000Z

@yourchanges Hi! This logs look normal: gpu inference worker just ask for new task for completion. Looks like you're using chat and it works? But I see no completions.

Answer 3 · 2024-02-15T03:34:02.000Z

@mitya52 Yes, it could be closed.
I found it's the vscode and idea plugin bugs, it not start the refact-lsp process successfully. so I manually start a refact-lsp process

refact-lsp --address-url https://xxx:8008/ --http-port 8001 --lsp-port 8002 --logs-stderr

and now, the chat and completions works perfect!!!

Thank you for your great work!

Answer 4 · 2024-02-15T09:09:13.000Z

@yourchanges please report this bug to refact-lsp repo

Answer 5 · 2024-02-15T10:02:12.000Z

Wait, 8001 and 8002 are debug ports. Are you sure @yourchanges you don't have xDebug on?

Answer 6 · 2024-02-15T13:31:19.000Z

@olegklimov for using the my own refact-lsp process which listen the 8001 and 8002, I have to changed the settings.json

{
    "workbench.colorTheme": "Default Light+",
    "go.toolsManagement.autoUpdate": true,
    "http.proxyStrictSSL": false,
    "http.proxyAuthorization": null,
    "refactai.pauseCompletion": false,
    "refactai.codeCompletionModel": "deepseek-coder/5.7b/mqa-base",
    "refactai.insecureSSL": true,
    "refactai.xDebug": true,
    "refactai.addressURL": "https://xxx:8008/",
    "refactai.apiKey": "any-will-work-for-local-server"
}

Answer 7 · 2024-02-15T13:37:47.000Z

@mitya52 I need some time to reproduce the bug, I always get the timeout error for connecting to http://127.0.0.1:904x , it almost caused by failure start the refact-lsp process, but I'm not familiar the vscode plugin developing, so I need some time to find how to debug the vscode plugin

Answer 8 · 2024-02-15T14:20:43.000Z

@mitya52 @olegklimov
By the way, I have another question:
I have two RTX 3090 24GB with nvlink, when I do a finetune it not use the two gpu. and the 5950 minutes is too long for one gpu.

so are there any tips/ways to do a finetune on two gpus or four gpus?

Answer 9 · 2024-02-19T18:37:02.000Z

@yourchanges unfortunately finetune process uses only one (cuda:0 device). But if you want to contribute and make possible to tune models in distribute way, you're welcome!

Answer 10 · 2024-02-20T09:07:59.000Z

@mitya52 thanks