Support Plugins on Client-Side
Livshitz opened this issue · 1 comments
Livshitz commented
We currently use LangchainJS (https://github.com/hwchase17/langchainjs) to load the AI-Plugins and process the response. Since it does not support client-side handling, we have to send the API keys to a node BE, let it process and get a response.
This has a few drawbacks:
- Playground shouldn't sent apikeys except directly to the LLM provider
- Agents in Langchain don't have proper control over external (HTTP requests, web pages) that are being used with the AiPlugin, causing overflow of text and tokens being sent to LLMs.
- Execution time is very slow
- No streaming
Ideally, we should have a solution where the selected plugins are being executed in client-side, and present the progress in the chat window step-by-step until getting the final result.