Weird GPU out of memory issue
fnrcum opened this issue · 2 comments
I'm encountering a relatively weird behavior when running the generator from the commandline using bark_perform.py
When running everything from the webui while using the OFFLOAD_CPU option, I have no issues generating the wav file for a long piece of text but when running it from the command line with the exact same options, I seem to constantly get the not enough vram problem.
python bark_perform.py --prompt_file prompts\news.txt --split_input_into_separate_prompts_by "line" --output_dir "output" --output_format "wav" --output_filename "news.wav" --history_prompt "D:\pythonProject1\speaker1.npz" --OFFLOAD_CPU "True" --USE_SMALL_MODELS "True" --show_generation_times "True" --text_temp "0.7" --waveform_temp "0.7" --output_iterations "1" --extra_stats "True" --text_use_gpu "True" --text_use_small "False" --coarse_use_gpu "True" --coarse_use_small "False" --fine_use_gpu "True" --fine_use_small "False" --codec_use_gpu "True" --split_character_goal_length "165" --split_character_max_length "205" --add_silence_between_segments "0.25"Keep in mind that I have also tried the OFFLOAD_CPU option as an environment variable and I see the same issue
$env:OFFLOAD_CPU="True"
Anyone have any ideas what I'm missing? I even tried using the small models and same issue. My videocard has 8GB VRAM
It's a bug in the CLI. Let me do a quick patch.
All set.
Do a git pull and it should work. I had left it hardcoded in the CLI actually it was just ignoring offloading.
BTW you only need python bark_perform.py --text_prompt "test" --OFFLOAD_CPU True and it defaults to on, you don't even need that. Large Models work perfect even in 6GB of memory with offloading. And they are worth it, they sound a lot better than small.