Hardware requirement for huggingface model

Question

Hardware requirement for huggingface model

Opened this issue 8 months ago · 3 comments

cheliu-computation commented 8 months ago

Thanks for your impressive work!

I check the demo code on the hugginface. It is written nice.
Could you suggest me the minimum hardware configuration for this demo? Such as 3090 or Colab GPU

Answer 1 · 2024-01-24T17:16:05.000Z

I use CPU to inference(about 50GB RAM). It works.

Answer 2 · 2024-01-24T17:19:34.000Z

You also need to change float16 to float32 for CPU inference.

Answer 3 · 2024-01-30T20:47:18.000Z

Hey @cheliu-computation and @dirtycomputer,

Thanks for your information @dirtycomputer !

We tested the demo code using A100-40G and will suggest the minimum hardware configuration after testing it.

Best,
Zhihong