海光服务器+paddle inference专用推理库+麒麟OS
Opened this issue · 4 comments
请提出你的问题 Please ask your question
请问Paddle专用推理库（paddle inference）是否已适配基于x86架构的国产海光服务器？我在官方文档中找到的相关资料较为有限且有些模糊，想确认一下具体情况。如果已适配有无具体的部署方案或Docker
你好，可以按照下面的步骤进行海光上的性能调优：
-
使用镜像：registry.baidubce.com/device/paddle-dcu:dtk24.04.1-kylinv10-gcc82，该镜像支持K100_AI的高性能推理，启动命令可以参考：docker run -it --name paddle-dcu -v
pwd:/ssd1
-w=/ssd1 --shm-size=128G --network=host --privileged
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined
registry.baidubce.com/device/paddle-dcu:dtk24.04.1-kylinv10-gcc82 /bin/bash -
安装Paddle: python -m pip install --pre paddlepaddle-dcu -i https://www.paddlepaddle.org.cn/packages/nightly/dcu/
-
执行推理程序
推理调优的一些建议：
-
我们内部对LLM+K100_AI做过一些调优，视觉类模型没有做过相关的调优，推荐可以使用profiler进行推理的性能分析，找到性能卡点，具体文档可以参考：https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api/paddle/profiler/Profiler_cn.html，https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/performance_improving/profiling_model.html，
对于推理来说，可以直接使用profiler.start()和profiler.stop()来进行性能分析； -
对于海光K100_AI来说，rocblas gemm不同转置配置下的性能：NT > NN > TN，可以重点关注下模型中GEMM的配置，PR可以参考：https://github.com/PaddlePaddle/PaddleNLP/pull/8800；获取GEMM配置的方式可以运行一次模型推理，并设置环境变量：export ROCBLAS_LAYER=3，从log中抓取gemm shape;
可以先按照上述方式尝试下，有分析结果后可以再发出来一起看下~
目前，我在安装paddlepaddle-dcu后
出现报错：
(test) (Tokto) (paddle) [root@localhost defectDetection20241204]# python main_test.py
Error: Can not import paddle core while this file exists: /root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/base/libpaddle.so
Traceback (most recent call last):
File "/root/doc/defectDetection20241204/main_test.py", line 4, in <module>
from data.tools import check_image
File "/root/doc/defectDetection20241204/data/tools.py", line 15, in <module>
from PaddleYOLO.tools import infer
File "/root/doc/defectDetection20241204/PaddleYOLO/tools/infer.py", line 33, in <module>
import paddle
File "/root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/__init__.py", line 37, in <module>
from .base import core # noqa: F401
File "/root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/base/__init__.py", line 38, in <module>
from . import ( # noqa: F401
File "/root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/base/backward.py", line 28, in <module>
from . import core, framework, log_helper, unique_name
File "/root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/base/core.py", line 380, in <module>
raise e
File "/root/anaconda3/envs/test/lib/python3.10/site-packages/paddle/base/core.py", line 267, in <module>
from . import libpaddle
ImportError: libgalaxyhip.so.5: cannot open shared object file: No such file or directory
(test) (Tokto) (paddle) [root@localhost defectDetection20241204]# python -m pip install paddlepaddle-dcu==3.4.2 -i https://www.paddlepaddle.org.cn/packages/stable/dcu/
Looking in indexes: https://www.paddlepaddle.org.cn/packages/stable/dcu/
ERROR: Could not find a version that satisfies the requirement paddlepaddle-dcu==3.4.2 (from versions: 3.0.0b0, 3.0.0b1, 3.0.0b2)
ERROR: No matching distribution found for paddlepaddle-dcu==3.4.2
(test) (Tokto) (paddle) [root@localhost defectDetection20241204]# pip list
Package Version
----------------- -----------
anyio 4.4.0
astor 0.8.1
certifi 2024.7.4
contourpy 1.3.1
cycler 0.12.1
decorator 5.1.1
exceptiongroup 1.2.2
fonttools 4.55.2
h11 0.14.0
httpcore 1.0.5
httpx 0.27.0
idna 3.7
kiwisolver 1.4.7
matplotlib 3.9.0
networkx 3.3
numpy 1.26.4
opencv-python 4.5.5.64
opt-einsum 3.3.0
packaging 24.2
paddlepaddle-dcu 3.0.0b2
pillow 10.3.0
pip 24.2
protobuf 5.27.2
pyparsing 3.2.0
python-dateutil 2.9.0.post0
PyYAML 6.0.1
setuptools 75.1.0
six 1.17.0
sniffio 1.3.1
typing_extensions 4.12.2
wheel 0.44.0
(test) (Tokto) (paddle) [root@localhost defectDetection20241204]#
docker pull iregistry.baidu-int.com/device/paddle-dcu:ubuntu22-clang15-py310 可以拉取这个镜像再安装paddle试试