OpenBMB/BMInf

AttributeError: type object 'cublasLt' has no attribute 'cublasLtHandle_t'

wac81 opened this issue · 3 comments

wac81 commented

Cuda compilation tools, release 10.0, V10.0.130

torch 1.6.0

python 3.6

and get error:

Traceback (most recent call last):
File "/home/wac/PycharmProjects/CPM-1-Generate/test.py", line 7, in
cpm2.generate(text)
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/models/cpm2.py", line 219, in generate
frequency_penalty, presence_penalty, 189
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/models/cpm2.py", line 103, in pre_processing
ctx = self.encode(np.array([idx], dtype=np.int64), [input_length])
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/arch/t5/model.py", line 238, in encode
True
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/layers/transformer_block.py", line 42, in forward
x = self.self_attention.forward(allocator, x, attention_mask, self_attn_position_bias)
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/layers/attention.py", line 63, in forward
qkv_i32
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/functions/gemm.py", line 86, in igemm
_igemm(allocator, a, aT, b, bT, c, device, stream)
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/functions/gemm.py", line 102, in _igemm
lthandle = get_handle(device)
File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/functions/gemm.py", line 65, in get_handle
v = cublasLt.cublasLtHandle_t()
AttributeError: type object 'cublasLt' has no attribute 'cublasLtHandle_t'

a710128 commented

BMInf requires CUDA version >= 10.1

wac81 commented

use cpm2.1
just try generate，but not good

input:
text = "圆圆的月儿天上挂，圆圆的月饼香天涯，圆圆的快乐美如花，圆圆的祝福到你家：美满日子玉润珠圆，幸福生活花好月圆，合家吉祥永团圆。 "

output:
('欢乐节日圆团建，每天健康、平安丽日长日久正，好好价钱苟得长辈羊肉，比肩为知友”，故以番语改编为庆生“。【美女同事】美人肯定是美的了，干脆用“爱司机”儿装个美女，然后自己夜不归宿，想着让家里人趁机感受。\n', True)

how to impove?

a710128 commented

use cpm2.1 just try generate，but not good

input:
text = "圆圆的月儿天上挂，圆圆的月饼香天涯，圆圆的快乐美如花，圆圆的祝福到你家：美满日子玉润珠圆，幸福生活花好月圆，合家吉祥永团圆。 "

output:
('欢乐节日圆团建，每天健康、平安丽日长日久正，好好价钱苟得长辈羊肉，比肩为知友”，故以番语改编为庆生“。【美女同事】美人肯定是美的了，干脆用“爱司机”儿装个美女，然后自己夜不归宿，想着让家里人趁机感受。\n', True)

how to impove?

After a period of testing, we found that using beam search was effective in improving the generation results. We consider to use beam search for text generation in later versions.

  • 👍1