ValueError: model.decoder.embed_tokens.weight doesn't have any device set.

[mwanzu]

i have downloaded the huge model and i get the above error when i run the example below
import galai as gal
model = gal.load_model("huge", 'float16', 1)
model.generate("Scaled dot product attention:\n\n\[", device_map="auto")

[mkardas]

Hi @mwanzu, can you check on galai 1.1.0? Also, with num_gpus=1 you'll get Out of Memory Exception, as the "huge" model weights are 240 GB in dtype=float16.