Limiting the response to the tables avb in the schema
MohamadOHijazi opened this issue · 4 comments
I have been performing multiple tests on the online Demo. I have noticed that the model is ignoring instructions and generating results for prompts that contain Tables different than the schema provided.
I used the same schema in the demo and tested it multiple times however the model always generated Queries targeting tables that are not included in the schema provided.
Is their a way to address this issue without applying another model to classify if the prompt is relative to any of the tables AVB and if the prompt is referring to non-existing attribute?
an example is when asking ordering sales person by age. the model generated A DML the model didn't recognize that age is not avb in the sqldabase schema.
I attached an example image about this issue bellow.
Hi there, thank you for opening this issue and for including a replicable example. This seems to be a model regression that escaped our unit tests. we'll update the model weights to address this soon.
@rishsriv Thanks for your fast reply! And for your amazing project.
After thoroughly searching it was the best open source LLM that I could find.
To solve this issue I was thinking of adding a filtering layer that drops irrelevant queries. However, this will increase computational power needed and will remove the edge of this model.
Looking forward to try your updated model.
This issue is stale because it has been open for 30 days with no activity.
This issue was closed because it has been inactive for 14 days since being marked as stale.