How to connect to backend?
Closed this issue · 3 comments
Hi, I was able to get the bot running and connected to discord, but I'm not sure where to put the api key or otherwise link it up to the LLM I'm running locally. Where/how do we input that info?
The only API key you need to add is a Discord API key in the bot.py file on line 19.
discord_api_key = "Place your Discord Bot API Key here"Other than that, you just load up the LLM of your choice inside koboldcpp. The bot expects the koboldcpp api server to be running on port 5001, which is the default for koboldcpp.
If it's not working, it may be a firewall isue, or an IP address error.
To change the port or IP address to the Koboldcpp LLM server, you'll find that on line three of the text-default.json file in the /cofigurations directory. The line looks like this:
"address": "http://localhost:5001/api/v1/",If for some reason it's not seeing Koboldcpp, and it's running on the default port, you may need to change the IP address to:
"address": "http://0.0.0.0:5001/api/v1/",or
"address": "http://127.0.0.1:5001/api/v1/",If you have koboldcpp running on a different machine, and/or a different port than the default, you wil need to change this line to match the correct IP address and port number.
Cheers!
I'd love to see instructions for using this with WebUI.
I'd love to see instructions for using this with WebUI.
The Oobabooga standard API is compatible with KoboldAI requests. Follow the above instructions, just change the address to Oobabooga's API address.
Cheers!