A template/demo to show how to use the godot-llm addon
You may simply download the Android apk in the release page to try it out for Android devices.
For Windows/Linux:
- Get the
Godot LLM Templatefrom the Godot asset library, or clone this project:git clone https://github.com/Adriankhl/godot-llm-template.git - Get
Godot LLMdirectly from the godot asset library, or download the addon from godot-llm and place it to theaddonsfolder - Download models
- Run the project with Godot or export an Android apk (requires the
Manage External Storagepermission)
- Text generation recommendation: Meta-Llama-3-8B-Instruct-Q5_K_M.gguf or Phi-3-mini-4k-instruct-Q2_K.gguf for low-end devices such as Android phone
- Similarity/Vector database recommendation: mxbai-embed-large-v1.Q5_K_M.gguf
- Image to text recommendation: llava-phi-3-mini-int4.gguf, also download the
llava-phi-3-mini-mmproj-f16.gguffrom the same page
Folder and file structure:
├── addons
│ └── godot_llm
│ ├── bin
│ │ ├── libgodot_llm.android.debug.aarch64.so
│ │ ├── libgodot_llm.android.release.aarch64.so
│ │ ├── libgodot_llm.linux.debug.x86_64.so
│ │ ├── libgodot_llm.linux.release.x86_64.so
│ │ ├── libgodot_llm.windows.debug.amd64.dll
│ │ └── libgodot_llm.windows.release.amd64.dll
│ └── godot_llm.gdextension
├── db.gd
├── db.tscn
├── icon.svg
├── icon.svg.import
├── image_to_text.gd
├── image_to_text.tscn
├── LICENSE
├── main.gd
├── main.tscn
├── media
├── project.godot
├── README.md
├── similarity.gd
├── similarity.tscn
├── text_generation.gd
└── text_generation.tscn
You can now open the application with Godot
You will see 4 modes: Text Generation, Similarity, Image to Text, and Vector Database.
In each mode, you need to click Model to point to the correct model gguf file. Additionally, in Image to Text mode, you need to click Mmproj to point to your *mmproj*gguf file.
You will see there are 3 generation mode: Simple, Instruct, and Interactive.
Just edit the prompt and click Generate. Click Stop to stop the generation.
You choose None schema or Person schema, if you choose None schema, the model will generate the information of a character with Json format.
Click Start first with empty prompt, then modify the prompt and click Continue when it is available to talk to the AI. This is an interactive mode. Whenever you see the Continue button is available, either input something in the prompt and click Continue to send the input, or simply click Continue to let the AI talk itself.
Similar to the instruct mode, except that it relies on a good initial prompt and additoinal reverse prompt and input suffix settings to generate a smooth conversation. The preset prompt should work decently well, click the Start button with the preset prompt and start talkin to the AI.
Input prompts to compute the Embedding, or input two prompts to calculate their Similarity
Select an Image from your device. Alternative, click From View to capture your game screen as the image. You can Compute Base64 encoding for your image, or decode a base64 string back to Compute Image. Edit the prompt, and generate text from Image, Base64, or View (your game screen).
Click Metadata: id, year to define the metadata of the tables, then click Create tables to create a some tables for embedding storage and retrieval. Click Store text to split the document into chunks and store into the database, you may also click Split text to see the chunks. Click Retrieve to retrive three most similar text chunks with the upper left prompt, which satisfy the sql-where-clause in the middle right. You may also turn on "Verbose stdout", modify the upper prompt to a valid sql statement, and click execute to run the sql statement directly, you will see the result in godot log.