How to integrate in actix-web?
ChaoII opened this issue · 5 comments
Hi,
I am trying to integrate in actix-web , but when I try to pack params and model to web::Data<....> . Could you please give me some demo to implement my idea? otherwise, how to use in multithreading avoid to block other web request?
See #86 for an idea on how to use the model and state. As for running without blocking, tokio has tokio::task::spawn_blocking for this exact purpose. Unsure what actix's equivalent would be.
thank you very much.
when I use mutithreading , something with wrong:
model: ggml-large-v3-q5_0.bin
GGML_ASSERT: C:\Users\AC\CLionProjects\whisper-rust\target\debug\build\whisper-rs-sys-27850de1f340e604\out\whisper.cpp\ggml-cuda.cu:6742: ptr == (void *) (g_cuda_pool_addr[device] + g_cuda_pool_used[device])
error: process didn't exit successfully: `target\debug\whisper-rust.exe` (exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)
@ChaoII, I use whisper across many threads without issue using this crate, even with multiple whisper context. Can you post more code than just the output error?
I just upgrade to the latest whisper-rs release which appears to use whispercpp 1.5.4. I'm running into the same error as above, which I believe is related to:
I'm wondering if bumping to 1.5.5 will resolve this given the last comment on the thread?
This may also be worth review:
Although it does seem strange to access the same context/ggml backend across multiple threads at the same time, it might be worth noting in the docs that context can't be used in this way without resulting in a crash.
Update on this:
Running against this pull request which updates to 1.5.5 solves the aforementioned issue for me.