paperswithcode/galai

Figure 3 explanation

Muennighoff opened this issue · 2 comments

Muennighoff commented

Sorry, a few things are unclear to me about Fig 3:

  1. What is generated by the model? Everything starting from <work>? Including Answer?
  2. Is it zero-shot like in the image?
  3. Is it a mistake from the model that it generates « & >?
  4. Does the model literally predict the output of the program (0.0051) or did you run the program externally and then add it to the image?

Thanks 🤗

Screenshot 2022-11-19 at 14 56 03

RJT1990 commented

Thanks for the comment: for inference, it generates everything after including Answer. It is "zero-shot" in the sense you just need to prompt with at test-time.

Muennighoff commented

Interesting, so the calculation 0.0051 was not done using an external program, but the model did it itself!
Thanks 👍