Enhancement idea: add /web function to search using jina
thiswillbeyourgithub opened this issue · 4 comments
Hi,
Jina recently unveiled their new search tool, it allows with a simple curl to get search result from the web in a LLM friendly format.
I was thinking you might be interested in this. Here's the link : https://jina.ai/news/jina-reader-for-search-grounding-to-improve-factuality-of-llms/
That would allow web search using any LLM given, although it would require a bit of engineering to avoid reaching the max_tokens limit or being too expensive for nothing. I can suggest implementation details for that if you want.
Have a nice day!
I think I will made such functionality paid along with speak while streaming. Remember that this app is open source and repository where you are posting your issue is only client app. I think jina search will need some backend for better performance so it is not in my plans for now. Thanks for understanding!
Also I receive no revenue from ads so I think I will replace ads with paid service. And if users will continue using my app without supporting (it not belongs to you) I will consider closing source code and making app paid.
I tested Jina and I found that search is not working (it always return 422 http error independently on user query). Reader works but works slowly.
The solution will be user prompt > Function call > Jina Search > Embeddings contextual search > LLM process > Output.
As for now response will be too long because jina nor embeddings API have sufficient performance.
So I'm considering replacing Jina with Perplexity and make it semi-paid (users can use Perplexity without API and pay me or users can set their API key).
This solution will be better because requests will have less response time.
Algorithm: User prompt > Function calling (will automatically determine when search is necessary) > If search is necessary perplexity will generate short summary > LLM process > Output.
This algorithm will be more precise, cheaper amd faster.
If you have other ideas you can start a discussion.
I tested Jina and I found that search is not working (it always return 422 http error independently on user query). Reader works but works slowly.
I think they are very new and were targeted by DDOS at the time. I would be surprised if they didn't get much faster pretty soon if not already. FWIW regarding what you're saying about response time : personnaly, I am completely willing to wait 30 seconds for a request if I know it's actually doing something that complex. But yeah perplexity will always be faster.
I have other things to say about the agorithm etc but want to think some more before bothering you