kohjingyu/gill

shape mismatch in the example "Multimodal Dialogue"

Closed this issue · 1 comments

I copy the whole jupyter code to the "test.py", and when I run the "Multimodal Dialogue" part, I encounter mistakes:

Traceback (most recent call last):
  File "test.py", line 130, in <module>
    full_outputs = generate_dialogue(prompts, num_words=num_words, sf=sf, temperature=temperature, top_p=top_p)
  File "test.py", line 67, in generate_dialogue
    return_outputs = model.generate_for_images_and_texts(
  File "/home/zhcheng/CoMT/Gill/gill/models.py", line 719, in generate_for_images_and_texts
    gen_emb = self.model.gen_text_hidden_fcs[0](raw_emb, gen_prefix_embs)  # (1, 77, 768)
  File "/home/zhcheng/anaconda3/envs/gill/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhcheng/CoMT/Gill/gill/layers.py", line 32, in forward
    x = x + input_embs
RuntimeError: The size of tensor a (0) must match the size of tensor b (8) at non-singleton dimension 1

It's the problem of the example image.