Idea: ChatGPT Coach for Student Conversations.

[edwardhamilton]

What would your feature do ?

Title: ChatGPT Coach for Student Conversations.

Objective:
To create a ChatGPT-based coach/moderator that can help parents and educators monitor and guide a student's interactions with ChatGPT. The moderator will be designed to provide insights into the student's interests and engagement with the conversation, as well as to guide the conversation in a direction that is more productive and educational.

Note: I'm actually doing this for a child who has some developmental issues. He is isolated and needs a window into the world. He loved using ChatGPT, but his parents found that he always got stuck asking the same questions. So the initial use case just requires nudging the conversations in the right direction.

Features:

• Topic Identification: topics will be identified and conversations will be grouped by similar topics

• Statistics: Statistics will be provided on the topics, how long the student is spending on each topic, and other relevant metrics.

• Conversation History: Each conversation history will be stored and linked to its topic. The coach will be able to navigate the topics and drill down to view conversation details. Coach will then be able to add specific topic level "instruction" to steer that topic discussion in some way.

• Coaching: By looking at topic statistics and conversation histories, the coach will gain insight into the student's interests and engagement with ChatGPT. The coach will then be able to steer the conversations in some way. Example - coach could ask ChatGPT to:

  1. Introduce a new topic or provide additional information on a particular subject.
  2. Don't allow child to talk about fantasy any more,  or too much.
  3. Make sure that student doesn't stay on the same topic for too long.
  4. Place time limits on ChatGPT usage (this is easy)
  5. Add a topic level instructions that will get inserted into the future new or continuing chats that are on this topic.
  

Implementation:

• Coaching: The ChatGPT coach will be implemented by silenting inserting / embedding coach's "instructions" into the individual chat history. So for example, at the beginning of a new chat, the app will silently insert instructions to chatGPT to not discuss fantasy. For topic level steerage / instructions, once the chat coach AI detects that a conversation is about a topic where the coach has specified topic level "instructions", these instructions will get silently inserted into the chat history, in order to steer the conversation.

• Content filtering: as we don't want the coach to interfere with the naturalness of the conversation, it may be necessary to filter out references to the coaching instruction from ChatGPT's responses. So for example: we don't want ChatGPT to say, "I can't discuss fantasy, because you already told me not to". So we might take the initialize response from ChatGPT, make a new ChatGPT request (on separate chat) and say "please remove any reference to coaches instructions" from the paragraph. There may be other use cases for content filter, for example if ChatGPT is not adequately responding to the "embedded" coach's instructions.

• Topics: are identified by having ChatGPT summarize each chat and then creating an embedding for each summary. The chat summary embeddings can then be fed to a clustering algorithm - which can be parameterized to determine the granularity of the topics. Once we have the topics, we will then further summarize all the chat summaries for that topic, to create a topic summary. Finally we'll ask ChatGPT to assign a topic name from the topic summary.

Currently I'm experimentation with this by manually adding prompts to ChatGPT to modify the subsequent conversation. I want to focus on the key aspects of the feature and don't want to have to learn how to implement a ChatGPT clone. I had originally thought about doing this with a Chrome Extensions, but the chat history is stored server-side. So it looks like I'll need a ChatGPT clone to do this. Then I'd have control of the complete chat history and be able to insert instructions as needed. Once I get the basic app running then I'd would do a lot of the research and testing.

Conclusion:
The ChatGPT-based coach/moderator for student conversations has the potential to be a valuable tool for parents and educators who want to monitor and guide their students' interactions with ChatGPT. By providing statistics, direct viewing of conversations, and the ability to request conversation changes, the moderator can help ensure that students are engaging with ChatGPT in a productive and educational way. With further development and testing, the ChatGPT-based moderator could become a widely used tool for enhancing student learning and engagement.

Proposed workflow

For the child/student (i.e. the user of ChatGPT). Nothing changes. Possibly the coach's instructions may be visible to the student. But it could be silent as well. Particularly for students with disabilities.
For the coach

1. Menu item to view the Conversation History.
2. Open the Conversation History
3. A list of topics will be displayed. With a summary, as well as statistics about how long student has been on that topic.
4. Coach can select a topic, and drill down into the summary, as well as the detailed conversations.
5. Coach can add usage constraints on the topic. (e.g. don't discuss this topic more than 1 hr per day). Coach can also add a "prompt" that would get inserted into a conversation when this topic is discussed.
6. Coach can add a "prompt" that gets inserted at the beginning of any conversation.

Ultimately, these are rough guidelines and I'll have to experiment to get it right.

Additional information

I'm willing to do much of the work on this, but just want to use as much of current project as possible.