replicate/cog-llama-template

Error: Inplace update to inference tensor outside InferenceMode is not allowed

Pwntus opened this issue · 0 comments

Pwntus commented

A repetition_penalty of 1.1 (or any custom value) will generate the following error:

Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See pytorch/rfcs#17 for more details.