Is there anyway to route the plugin to a privateGPT instance hosted locally on the same computer or on a local network?

Question

Is there anyway to route the plugin to a privateGPT instance hosted locally on the same computer or on a local network?

dziliak opened this issue a year ago · 2 comments

I love the idea of using this plugin on an offline LLM instead of giving my data to the cloud. Are there any suggestions on where to look in this code and other resources to kludge something together to use privateGPT instead of openAI?

I didn't see anyone else asking this and hope this is the right spot to ask it.

Answer 1 · 2024-01-26T04:34:56.000Z

Seconded but for something like Oobabooga (or Ollama once they get on windows)... or stronger integration with GPT4All if their limits can be tolerated. Not sure about LocalChat and Local AI
Same applies to debanjandhar12/logseq-chatgpt-plugin#32

Answer 2 · 2024-03-27T21:38:44.000Z

llama.cpp's server provides a more or less openai compatible api, so making the api url configurable might be all thats needed.