Model uses wrong syntax for PostgreSQL queries in amazon-bedrock-amazon-rds-poc
felipeladik opened this issue · 5 comments
Questions generated by the AI model are using the wrong PostgreSQL syntax for string literals. Example:
psycopg2.errors.UndefinedColumn: column "Untitled (page from Sump)" does not exist
LINE 1: ...artist_id = T2.artist_id WHERE T2.title = "Untitled ...
String literals should be wrapped around single quotes.
Hey @felipeladik what model are you using to generate the SQL query?
Hi @ryanadoty, I used the default settings from the POC. It was using amazon.titan-text-express-v1 model.
@felipeladik , What question did you ask when you received this error?
My guess is that it was a more complex query that needed to be constructed and it was just a bad output of the model...
This is the primary challenge with natural language to SQL based use cases is that at times the underlying LLM you are using can struggle to assemble a valid SQL query, resulting in an error such as this one.
@ryanadoty wasn't anything really complex. Basically any question that would result into a where = 'something' would cause that issue. For example I would ask what was the heaviest artwork in the database and then who is the artist of that artwork.
Hey @felipeladik, this can likely be fixed by further prompt engineering as well as using other models but is not something functionally wrong with the POC itself...
With that said if you wanted to tweak the POC to better handle these types of queries, adding sample prompts in a similar format as the sample prompts in this file, you can help reproduce more consistent, correct results for these types of queries: https://github.com/aws-samples/genai-quickstart-pocs/blob/main/genai-quickstart-pocs-python/amazon-bedrock-amazon-rds-poc/SampleData/moma_examples.yaml