Minimum vram?

[ais69]

What's the min vram for this? Thanks

[wenqsun]

Thanks for your interest! The requirements for our project are same as CogVideoX, your can refer this link: https://github.com/THUDM/CogVideo/tree/main

Specifically, the min vram for our project can be lower than 10GB if you put the model to the cpu following the operation: https://github.com/THUDM/CogVideo/blob/e2987ff565703953b34749db2d1053e26bba2e2c/inference/cli_demo.py#L101

[mp3pintyo]

For me, the 4090 card failed with an OOM error. It worked perfectly well on the A40 card (48 GByte VRAM).

[wenqsun]

If you want to run on 4090, then you are required to put the model to the CPU, leading to a longer generation process. Have you tried the code below to reduce the memory requirement:

# pipe.to("cuda")
pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

[mp3pintyo]

I tried to use vae slicing because I noticed that there is a sudden memory jump at the very end of the video generation. Unfortunately, the python error message said that this cannot be used for this model. Maybe because cuda usage was turned on and I tried it that way.
In any case, since it needs so much memory anyway, it is easier to run it on Runpod on an A40 than on your own computer. :)

[wenqsun]

Yes, it would be better to run on an A40.

[mp3pintyo]

I pasted it into the code, but for some reason it doesn't want to work in a RunPod environment with a 4090 card. No matter, the A40 remains.

[linghu-cell]

A6000 48G It will explode the video memory？

[mp3pintyo]

48 is plenty. It also runs perfectly on the A40, which has a similar amount of memory.