Value error occurs when trying to pass "already_preprocess true" for the training

[ssoyeonn]

Hi, I tried to train llama-3 with the training data you provided but it seems like there is something wrong with the already_preprocess argument.

I get the following error message:
[2024-07-15 22:48:32,516] torch.distributed.run: [WARNING]
[2024-07-15 22:48:32,516] torch.distributed.run: [WARNING] *****************************************
[2024-07-15 22:48:32,516] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2024-07-15 22:48:32,516] torch.distributed.run: [WARNING] *****************************************
/home/common/miniconda3/envs/usr/lib/python3.8/site-packages/transformers/training_args.py:1693: FutureWarning: using -fsdp_transformer_layer_cls_to_wrap is deprecated. Use fsdp_config instead
warnings.warn(
Traceback (most recent call last):
File "fastchat/train/train_mem.py", line 13, in
train()
File "/home/common/miniconda3/envs/usr/lib/python3.8/site-packages/fastchat/train/train.py", line 262, in train
model_args, data_args, training_args = parser.parse_args_into_dataclasses()
File "/home/common/miniconda3/envs/usr/lib/python3.8/site-packages/transformers/hf_argparser.py", line 347, in parse_args_into_dataclasses
raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--already_preprocess', 'True']

But when I don't pass the argument "--already_preprocess True", then I also get an error message asking for "conversations"
inside the file as in this issue :
#7

I downloaded the training data that you generated and converted the jsonl file in the "prompted" directory into json file (since I've got a JSONDecodeError when I tried to use the jsonl file directly as the training data).

The json file looks like this:

{"prompt": "I want you to act like Ludwig van Beethoven. I want you to respond and answer like Ludwig van Beethoven, using the tone, manner and vocabulary Ludwig van Beethoven would use. You must know all of the knowledge of Ludwig van Beethoven. \n\nThe status of you is as follows:\nLocation: Vienna\nStatus: Beethoven, the famous composer, is in Vienna to study with Joseph Haydn, another renowned composer. Beethoven had met Haydn a couple of times in Bonn before, and arrangements were made for him to study with the master in Vienna. Beethoven is filled with admiration for Mozart, and he wants to express this to Haydn.\n\nThe interactions are as follows:\n\n", "output": "Beethoven (thinking): I cannot wait to tell Haydn about my love for Mozart. I hope he understands how much I admire him and his works. I believe Haydn can help me learn more about Mozart's spirit, and I am excited to start my lessons.<|eot|>\nBeethoven (speaking): Good afternoon, Haydn. It's great to see you again.<|eot|>\nHaydn (speaking): Good afternoon, Beethoven. It's good to see you too. How have you been?<|eot|>\nBeethoven (speaking): I have been well, thank you. I have been looking forward to studying with you and learning more about music.<|eot|>\nHaydn (speaking): That's good to hear. I am excited to teach you and see what you can do.<|eot|>\nBeethoven (speaking): I also wanted to talk to you about Mozart. I have always been a great admirer of his works, and I believe he was a genius. I would love to learn more about his spirit and style.<|eot|>\nHaydn (speaking): I agree with you, Beethoven. Mozart was a great composer, and his works have inspired many musicians, including me. I would be happy to teach you about his style and help you understand his music better.<|eot|>\nBeethoven (thinking): I am glad Haydn understands my love for Mozart. I hope he can teach me more about his works and style. I am excited to learn from him and become a better musician.<|eot|>\nBeethoven (speaking): Thank you, Haydn. I am excited to start my lessons and learn from you.<|eot|>\nHaydn (speaking): You're welcome, Beethoven. Let's get started then.<|eot|>", "source": "seed_dialogue_6"}

Here is the training script:

torchrun --nproc_per_node=8
--master_port=20031 fastchat/train/train_mem.py
--model_name_or_path /path/to/llama3-8b/models--Undi95--Meta-Llama-3-8B-hf/snapshots/78fb9e22c486cec013a2109bd0bdddc6925369bb
--data_path /path/to/train_data/prompted/json/prompted_agent_dialogue_$CHARACTER.json
--already_preprocess True
--bf16 True
--output_dir /path/to/llama3-8b/ckpt/${CHARACTER}_7b
--num_train_epochs 10
--per_device_train_batch_size 2
--per_device_eval_batch_size 16
--gradient_accumulation_steps 4
--evaluation_strategy epoch
--save_strategy epoch
--save_total_limit 10
--learning_rate 2e-5
--weight_decay 0.1
--warmup_ratio 0.04
--lr_scheduler_type cosine
--logging_steps 1
--fsdp 'full_shard auto_wrap'
--fsdp_transformer_layer_cls_to_wrap LlamaDecoderLayer
--tf32 True
--model_max_length 2048
--gradient_checkpointing True

Would you please tell me which version of transformers you used? Do you know any other resolution?
Thank you.

[choosewhatulike]

Hi, thanks for raising this issue!
We use transformers==4.34.0. But I think this is related to FastChat. It seems that you are using the original fastchat package instead of the modified one in https://github.com/choosewhatulike/trainable-agents/tree/main/FastChat.
We changed the dataset loader in the training script of FastChat to support the data format of Character-LLM. You can use this version for training to see if it works.

[ssoyeonn]

Thanks a lot! I'll try that.