Redesign llamabot underneath the hood

[ericmjl]

It's dawning on me that SimpleBot is the modular core of our bots, and that the following capabilities are modular add-ons:

• Chat, within ChatBot
• RAG, which lives within QueryBot and is built on top of llama_index
• Function calling, which is not implemented yet but is on my list to do so

Thinking about these as modular capabilities gives me a path forward for internal refactoring of llamabot. I'd imagine the following components in a rewrite, so that we better match emerging paradigms of organizing LLM components:

• Bot module
• History + document storage
• Retrieval methods (e.g. last k from history, or vector similarity)
• Function-calling module (have yet to implement this)

The Bot module is the interface with LLMs underneath the hood. SimpleBot follows this paradigm.

The history module is generalized by "document storage". I'm imagining at least the following types of document stores:

1. a simple chronological history-like document storage, where the order of documents matters,
2. a llama_index-like vector storage system for vector similarity search

ChatBot is nothing more than a document module that can be composed with SimpleBot, and the addition of a chronological last-k retrieval method. The document module gets updated with every chat entry.

QueryBot is nothing more than a document module + vector similarity retrieval module that is composed with SimpleBot.

One can imagine a RAGChatBot, then, that uses ChatBot's document module but pairs it also with the vector similarity retrieval module.

Function calling, at the moment, is best supported through the OpenAI LLMs, but can be freely composed in as well.

What's not clear to me right now is how to design the interfaces between the components. I think this will need further thought.

(To be clear, I doubt I'm the first to think of this. It's just becoming clear in my head, that's all 😄.)

[ericmjl]

This is done :)