towhee-io/towhee

[Bug]: 在多次使用 pipeline('towhee/audio-embedding-vggish') 时发生内存泄漏

wangdabee opened this issue · 11 comments

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

embedding_pipeline = pipeline('towhee/audio-embedding-vggish')
outs1 = embedding_pipeline(wav_path)

在循环使用上述代码时,发生内存泄漏问题。达到20%内存

Expected Behavior

No response

Steps To Reproduce

No response

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):0.9.0
- OS(Ubuntu or CentOS):
- CPU/Memory:
- GPU:
- Others:

Anything else?

No response

此外,在循环使用代码时,会出现卡住的现象,即程序没有进行任何操作,也不报错。ctrl c 强制退出时 在 threading 相关代码处停止。

Upgrade Towhee to the latest version 1.1.1, and try it: https://towhee.io/towhee/audio-embedding

升级到1.1.1后 循环使用代码时,程序卡住的现象没有了,还是会有内存泄漏的现象,不过比0.9.0泄露的少一点

I have run this audio:https://towhee.io/towhee/audio-embedding/resolve/branch/main/test.wav 10,000 times. The memory usage has remained stable at around 2.9GB. I have not observed any memory leaks.

path1 = '1.wav'
path2 = '2.wav'

p = AutoPipes.pipeline('audio-embedding-vggish')

for i in range(1000):
res1 = p(path1)
res2 = p(path2)

res1 = res1.get()[0]
res2 = res2.get()[0]

运行以上代码,对源码的唯一修改是将device指定为cpu,内存占用大概12GB.
还有一个小问题,在运行完最后一次循环后,代码没有自动退出,一直卡住不动,需要手动ctrl c

image
I used this code snippet for testing, the memory was stable, and the problem could not be reproduced.

ok thank you for your patience

@junjiejiangjjj 您能说下您测试时使用的python版本吗

@junjiejiangjjj 您能说下您测试时使用的python版本吗

3.9.12

@junjiejiangjjj 解决了 我用的是python 3.7 发生的泄露