randaller/llama-chat

How to generate Bible data to LLAMA?

paulocoutinhox opened this issue · 4 comments

paulocoutinhox commented

Hi,

To a more real scenario, if i want input all the bible text into the LLAMA, how can i reach it?

Example of bible data:
https://raw.githubusercontent.com/tushortz/variety-bible-text/master/bibles/kjv.txt

Thanks.

randaller commented

Hi @paulocoutinhox!

We may train HF version with Bible data:

  • prepare datasets/bible.csv file from provided kjv.txt file, according to another /datasets examples
text
In the beginning God created the heaven and the earth. -- genesis 1:1
And the earth was without form, and void; and darkness was upon the face of the deep. And the Spirit of God moved upon the face of the waters. -- genesis 1:2
And God said, Let there be light: and there was light. -- genesis 1:3
And God saw the light, that it was good: and God divided the light from the darkness. -- genesis 1:4
...

modify hf-training-example.py

MODEL = 'decapoda-research/llama-7b-hf'
DATA_FILE_PATH = 'datasets/bible.csv'
OUTPUT_DIR = './trained'

run training

python hf-training-example.py

After such a long training time, I hope, LLaMA model probably will be able to supply us with a new AI commandments.

The prompt in hf-inference-example.py may be: "And "

paulocoutinhox commented

This is my other question, is this algorithm just for inference or can I use it for gpt chat style Bible questions?

randaller commented

@paulocoutinhox chat is just an imitation of chat, really both are just inference, chat is just when 2048 last tokens of dialogue with LLaMA has been passed as prompt for further inference. You may ask questions to LLaMA using just prompt easily. I'll try to add HF chat example soon.