VinAIResearch/PhoGPT

Evaluation lack of context

vTuanpham opened this issue · 3 comments

image

Is the question suppose to be this lack of context ? Given that only the Ground truth even mention about Vietnam.

I don't quite understand your question.
Please could you elaborate a bit more?

Hi @datquocnguyen . what i meant is that the question doesn't explicitly mention Vietnam, so it's reasonable to assume that it pertains to a global context rather than being specific to Vietnam. This is evidenced by the failure of all models to provide satisfactory answers to both questions.

It should be reasonable to assume that the question asks about Vietnam rather than the global context, as it's written in Vietnamese. You can think about other questions such as "vượt đèn đỏ phạt bao nhiêu tiền." If you'd like to ask about the global context, you should specify that in the input prompt.