[Bug]: Text to Video - tensors on cpu and cuda:0

[Craulback]

What happened?

Not sure what's on the CPU or how to move it to the GPU

Version

0.0.1 (Default)

What browsers are you seeing the problem on?

Firefox, Chrome

Where are you running the webui?

Windows

Custom settings

No response

Relevant log output

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper__index_select)
2022-11-19 15:30:33.666 Uncaught app exception
Traceback (most recent call last):
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 563, in _run_script
    exec(code, module.__dict__)
  File "J:\Downloads\sygil-webui-dev\installer\scripts\webui_streamlit.py", line 203, in <module>
    layout()
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\loguru\_logger.py", line 1226, in catch_wrapper
    return function(*args, **kwargs)
  File "J:\Downloads\sygil-webui-dev\installer\scripts\webui_streamlit.py", line 148, in layout
    layout()
  File "scripts\txt2vid.py", line 1899, in layout
    video, seed, info, stats = txt2vid(prompts=prompt, gpu=st.session_state["defaults"].general.gpu,
  File "scripts\txt2vid.py", line 1386, in txt2vid
    cond_embeddings = server_state["pipe"].text_encoder(text_input.input_ids.to(torch_device) )[0]
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\transformers\models\clip\modeling_clip.py", line 722, in forward
    return self.text_model(
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\transformers\models\clip\modeling_clip.py", line 632, in forward
    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\transformers\models\clip\modeling_clip.py", line 165, in forward
    inputs_embeds = self.token_embedding(input_ids)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\modules\sparse.py", line 158, in forward
    return F.embedding(
  File "J:\Downloads\sygil-webui-dev\installer\installer_files\env\envs\ldm\lib\site-packages\torch\nn\functional.py", line 2183, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper__index_select)

Code of Conduct

• I agree to follow this project's Code of Conduct

[Craulback]

Found a workaround on this repos Discord server.

Add the feature_extractor directory from https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main to the models\diffusers\stable-diffusion-v1-5 directory. And I removed the safety_checker entry from model_index.json
Thanks Strumpfsadin!

If you do this, I'd keep a backup of model_index.json to use with Text to Image, if you want the safety_checker disabled for that.