怎么实现多卡多进程预测，关于use_gpu : true写在config外层不能正常启动服务

Question

怎么实现多卡多进程预测，关于use_gpu : true写在config外层不能正常启动服务

JeremyGe07 opened this issue 9 months ago · 1 comments

PaddleHub2.3.1，PaddlePaddle2.4.2.post112，python3.7
您好，请问我按照下面的这个评论
#1726 (comment)
设置了一模一样的config，如下
{
"modules_info": {
"ocr_system": {
"init_args": {
"version": "1.0.0",
"use_gpu": true
},
"predict_args": {
}
}
},
"port": 8868,
"use_multiprocess": false,
"use_gpu": true,
"gpu": "0,1,2,3"
}

为什么hub serving start时，显示内容如下
599329892daa /paddle/gjz/PaddleOCR hub serving start -c deploy/hubserving/ocr_system/config.json
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
[2023-08-15 11:04:08 +0000] [15418] [INFO] Starting gunicorn 21.2.0
[2023-08-15 11:04:08 +0000] [15418] [INFO] Listening at: http://0.0.0.0:8868 (15418)
[2023-08-15 11:04:08 +0000] [15418] [INFO] Using worker: sync
[2023-08-15 11:04:08 +0000] [15423] [INFO] Booting worker with pid: 15423
[2023-08-15 11:04:08 +0000] [15436] [INFO] Booting worker with pid: 15436
[2023-08-15 11:04:08 +0000] [15454] [INFO] Booting worker with pid: 15454
[2023-08-15 11:04:08 +0000] [15456] [INFO] Booting worker with pid: 15456
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
use gpu: True
CUDA_VISIBLE_DEVICES: 1
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
use gpu: True
CUDA_VISIBLE_DEVICES: 3
use gpu: True
CUDA_VISIBLE_DEVICES: 0
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
use gpu: True
CUDA_VISIBLE_DEVICES: 2
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script`

向其发送请求返回的错误如下：
599329892daa /paddle/gjz/PaddleOCR python3 tools/test_hubserving.py --server_url=http://127.0.0.1:8868/predict/ocr_system --image_dir=./doc/64pic/ --visualize=false
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
[2023/08/15 11:04:33] ppocr INFO: Predict time of ./doc/64pic/1.jpg: 0.041s
Traceback (most recent call last):
File "tools/test_hubserving.py", line 158, in
main(args)
File "tools/test_hubserving.py", line 119, in main
res = r.json()["results"][0]
IndexError: string index out of range
————————————————————————————————————————————————————————

而正常只设置单卡时成功启动服务的输出如下：
λ 599329892daa /paddle/gjz/PaddleOCR hub serving start -c deploy/hubserving/ocr_system/config.json
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
[2023-08-15 11:12:20,073] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the init() to handle the initialization of the object
use gpu: True
CUDA_VISIBLE_DEVICES: 1
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script

Serving Flask app "paddlehub.serving.app_compat" (lazy loading)
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: off
Running on http://0.0.0.0:8868/ (Press CTRL+C to quit)`

————————————————————————————————————————————————————————

而且我发现当我把config中的use_gpu : true写在最外层，即使是单卡，也会不能成功启动服务，输出如下
λ 599329892daa /paddle/gjz/PaddleOCR hub serving start -c deploy/hubserving/ocr_system/config.json
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
[2023-08-15 11:11:19 +0000] [15953] [INFO] Starting gunicorn 21.2.0
[2023-08-15 11:11:19 +0000] [15953] [INFO] Listening at: http://0.0.0.0:8868 (15953)
[2023-08-15 11:11:19 +0000] [15953] [INFO] Using worker: sync
[2023-08-15 11:11:19 +0000] [15959] [INFO] Booting worker with pid: 15959
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
use gpu: True
CUDA_VISIBLE_DEVICES: 1
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script

只有把在外层的use_gpu : true 去掉，才可以正常启动服务。那么，该怎样实现多卡多进程呢？

Answer 1 · 2024-03-18T03:58:32.000Z

PaddleHub停止更新了，更多OCR的的预测流程，请参考：https://github.com/PaddlePaddle/PaddleOCR