Internet search is very slow using orca mini on 4bit in google colab T4 on gradio

Question

Opened this issue a year ago · 3 comments

Answer 1 · 2023-07-26T08:30:28.000Z

i waited for more than 15 minutes and didnt get the response

Answer 2 · 2023-07-26T09:57:29.000Z

which orcamini did you choose? 7B or 13B?

Answer 3 · 2023-07-27T09:52:50.000Z

i chose 13B and loaded it in 4bit