pingcap/ossinsight

[Explorer] Generate SQL failed for question 1c32bcd9-a63f-4572-938c-c5d44121516e: Which countries & regions led the database contributions in 2023?

ngaut opened this issue · 1 comments

ngaut commented

Hi, I have some problems with the question 1c32bcd9-a63f-4572-938c-c5d44121516e Which countries & regions led the database contributions in 2023? (errorType = error-sql-can-not-answer):

Error message

Failed to generate SQL, the question may exceed the scope of what can be answered.

The cause of the error may be a change in GPT's understanding of the semantics of the sqlCanAnswer field after the model was switched from the gpt-turbo-3.5-0301 to the gpt-turbo-3.5-0613 version. This has been resolved for now by adding more specific prompt descriptions:

https://github.com/pingcap-inc/ossinsight-ai/blob/143b08b23eeeb8f8ddaad16bce2372a6a30967e1/configs/prompts/explorer-generate-answer-with-context/template.mustache#L66

Rerun and it works again:

https://ossinsight.io/explore/?id=ea33bea9-7d32-4808-9173-b4ca743e0a8e