ParisNeo/lollms-webui

Upgrades for Database - Functional Export, and Import for ChatGPT

signalprime opened this issue · 5 comments

signalprime commented

If it plays a role, I'm testing with:
Binding: c_transformers, Model: mpt-30b-instruct.ggmlv0.q5_0.bin

I noticed the export functionality isn't working in the latest updates. I'm new to the project so I don't know when it last worked but it currently only exports an empty dictionary in the form of a text file containing "{}". Also the need to import data exported from ChatGPT is kinda essential.

So for the moment this is a placeholder to start the process of fixing the export process and more importantly (IMO) upgrading the database import so that we can move our ChatGPT exports into the database. I'll update this initial post with details throughout the coming days and welcome all input in the process.

Import Details as described here

The import process currently expects:

{ "id" : "discussion_id",
  "title": "discussion_title",
  "messages": "messages_data in list format"
}

The "messages" key expects list entries to contain the following items: sender, content, type, rank, parent, binding, model, personality, created_at, finished_generating_at, discussion_id

Export Details as described here

To be updated

ParisNeo commented

Thanks for your message
This has been fixed in last commit.
I'll think about adding import from chatgpt files. Maybe I'll add it later.

  • 👍1
signalprime commented

Thanks for your comments. It would definitely bring a lot of value to the project if users could import their ChatGPT data. A simple python script to adjust the ChatGPT JSON export to the format expected by LLOLMS will do the trick. I should have opened this as a discussion, perhaps we can move this issue to discussions. I'm searching for how

ParisNeo commented

I have built the script. Now i need to add a button to the ui. But this is getting too much work as people are asking for many features, like exllama support. So i need to prioritize

signalprime commented

Since you already have the script, maybe this pull request will help with that some? Thanks

ParisNeo commented

Thanks, I've accepted the PR. I'll integrate the script as soon as I can.

  • 👍1