Bug: AI hallucination

[a0m0rajab]

Describe the bug

I tried to find a contributor who used 3 different technologies, StarSearch has provided me some data about the contributors, yet one of them have not been active on GitHub for the last few years..

https://github.com/ojkwon

Steps to reproduce

1. go to star search
2. use the next prompt: give me three contributors who worked on all of TailwindCSS, NextJS and TRPC

[github-actions]

Thanks for the issue, our team will look into it as soon as possible! If you would like to work on this issue, please wait for us to decide if it's ready. The issue will be ready to work on once we remove the "needs triage" label.

To claim an issue that does not have the "needs triage" label, please leave a comment that says ".take". If you have any questions, please reach out to us on Discord or follow up on the issue itself.

For full info on how to contribute, please check out our contributors guide.

[nickytonline]

Hallucinations do happen and we have a disclaimer about this at the bottom of the StarSearch page. I'm going to close this for now as there's the disclaimer.

[a0m0rajab]

Thanks for the fast feedback!

[bdougie]

@a0m0rajab did you use the feedback widget after getting the response?

That is helpful for us to flag prompts for improvement.

Also would not consider this a hallucination, but an ungrounded result. The question is broad, so the result is prone to an ungrounded result.

[a0m0rajab]

@bdougie if you mean the thumb up/down, then no i did not use it. Will be careful next time and use that.

TIL: the meaning of ungrounded result!

[bdougie]

No worries. Just pointing out that exists and is very helpful for training the model.

[a0m0rajab]

Thanks for pointing that out! will use it next time.

[jpmcb]

Thanks for surfacing this: very helpful information as we continue to improve this. Abit more context on what's going on behind the scenes:

A question like give me three contributors who worked on all of TailwindCSS, NextJS and TRPC essentially has 3 subjects to search across: Tailwind, next, and trpc. We could do 3 vector searches for those 3 technologies but that wouldn't necessarily return the same individuals. So, you'd end up having to do a huge search where there is overlap across all 3 vector spaces for the same users which would take awhile.

It looks like what the model decides to do is one big vector search for any context that includes all 3 of those. And it popped up someone who isn't currently active resulting in an ungrounded answer.

It's possible it decided to use Bing search too which can pop up some ungrounded results. I'll dig more into this to see if there's a path forward for unraveling a better result.

[a0m0rajab]

Thanks for the explanation, based on that would not the model be able to alert/tell the user that it would not be able to search the three results? A simple check would be to have a prompt manager that can see the number of topic wthat searched for and tell the user it exceeds the maximum number (there is a paper from Microsoft about this I will try to find it when I am on pc (hopefully I won't forget that 😅)) Another approach that would be helpful to use is to show the user what the model actually doing (something like Devin and open Devin). It shows the planning and thinking process to the user. As for the users of these technologies, If you are using vectors multiplying the vectors would help to find the target user. Something like dot multiple. Sent from mobile device

…

________________________________ From: John McBride ***@***.***> Sent: Monday, May 27, 2024 5:33:16 PM To: open-sauced/app ***@***.***> Cc: Abdurrahman Rajab ***@***.***>; Mention ***@***.***> Subject: Re: [open-sauced/app] Bug: AI hallucination (Issue #3479) Thanks for surfacing this: very helpful information as we continue to improve this. Abit more context on what's going on behind the scenes: A question like give me three contributors who worked on all of TailwindCSS, NextJS and TRPC essentially has 3 subjects to search across: Tailwind, next, and trpc. We could do 3 vector searches for those 3 technologies but that wouldn't necessarily return the same individuals. So, you'd end up having to do a huge search where there is overlap across all 3 vector spaces for the same users which would take awhile. It looks like what the model decides to do is one big vector search for any context that includes all 3 of those. And it popped up someone who isn't currently active resulting in an ungrounded answer. It's possible it decided to use Bing search too which can pop up some ungrounded results. I'll dig more into this to see if there's a path forward for unraveling a better result. — Reply to this email directly, view it on GitHub<#3479 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AELNMKLEV7YKWRLLLHMRKSLZEM72ZAVCNFSM6AAAAABIJYOU4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZTGYYDMOJZGA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[a0m0rajab]

Note: I though about this question based on the T3 stack and decided to choose three technologies from it. Sent from mobile device

…

________________________________ From: Abdurrahman RAJAB ***@***.***> Sent: Monday, May 27, 2024 5:55:08 PM To: open-sauced/app ***@***.***>; open-sauced/app ***@***.***> Cc: Mention ***@***.***> Subject: Re: [open-sauced/app] Bug: AI hallucination (Issue #3479) Thanks for the explanation, based on that would not the model be able to alert/tell the user that it would not be able to search the three results? A simple check would be to have a prompt manager that can see the number of topic wthat searched for and tell the user it exceeds the maximum number (there is a paper from Microsoft about this I will try to find it when I am on pc (hopefully I won't forget that ??)) Another approach that would be helpful to use is to show the user what the model actually doing (something like Devin and open Devin). It shows the planning and thinking process to the user. As for the users of these technologies, If you are using vectors multiplying the vectors would help to find the target user. Something like dot multiple. Sent from mobile device

________________________________ From: John McBride ***@***.***> Sent: Monday, May 27, 2024 5:33:16 PM To: open-sauced/app ***@***.***> Cc: Abdurrahman Rajab ***@***.***>; Mention ***@***.***> Subject: Re: [open-sauced/app] Bug: AI hallucination (Issue #3479) Thanks for surfacing this: very helpful information as we continue to improve this. Abit more context on what's going on behind the scenes: A question like give me three contributors who worked on all of TailwindCSS, NextJS and TRPC essentially has 3 subjects to search across: Tailwind, next, and trpc. We could do 3 vector searches for those 3 technologies but that wouldn't necessarily return the same individuals. So, you'd end up having to do a huge search where there is overlap across all 3 vector spaces for the same users which would take awhile. It looks like what the model decides to do is one big vector search for any context that includes all 3 of those. And it popped up someone who isn't currently active resulting in an ungrounded answer. It's possible it decided to use Bing search too which can pop up some ungrounded results. I'll dig more into this to see if there's a path forward for unraveling a better result. — Reply to this email directly, view it on GitHub<#3479 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AELNMKLEV7YKWRLLLHMRKSLZEM72ZAVCNFSM6AAAAABIJYOU4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZTGYYDMOJZGA>. You are receiving this because you were mentioned.Message ID: ***@***.***>