open-sauced/app

Bug: prompt injection attack detector is not totally correct

Opened this issue · 4 comments

a0m0rajab commented

Describe the bug

I am getting 400 bad request error every now and then when I use the next prompt:

who is the best developer who knows TailwindCSS and used nextjs in his work?

www_screencapture_com_2024-5-26_16_00.mp4

Steps to reproduce

Method 1:

  1. go to StarSearch
  2. Try to ask few questions then ask: who is the best developer who knows TailwindCSS and used nextjs in his work?

Method 2:

  1. go to StarSearch
  2. ask who is the best developer who knows TailwindCSS and used nextjs in his work?
  3. Repeat the question
github-actions commented

Thanks for the issue, our team will look into it as soon as possible! If you would like to work on this issue, please wait for us to decide if it's ready. The issue will be ready to work on once we remove the "needs triage" label.

To claim an issue that does not have the "needs triage" label, please leave a comment that says ".take". If you have any questions, please reach out to us on Discord or follow up on the issue itself.

For full info on how to contribute, please check out our contributors guide.

nickytonline commented

I wasn't able to reproduced this issue with either method. See attached videos.

CleanShot.2024-05-27.at.16.56.40.mp4
CleanShot.2024-05-27.at.16.52.39.mp4
a0m0rajab commented

I tried this again and was able to reproduce the issue from the first time I asked the question:

image

jpmcb commented

I'm able to reproduce - It seems the in his work? part of your question is what's causing it to reject the prompt outright: it's probably being too aggressive. Behind the scenes, we use another AI agent to detect any malicious usage of the service.

Thanks for raising this - very useful and helpful information as we continue to iron out rough edges. I'll tackle looking at this 🕵🏻‍♂️