[BUG] <title> Qlora无法训练VL。 显示： we only spport quantization for text model. Support for vision, speech and multimodel will come later.

[xiayq1]

we only spport quantization for text model. Support for vision, speech and multimodel will come later.

无法使用qlora训练VL。

期望行为 | Expected Behavior

但是readme里面写的是：
2023.9.12 😃😃😃 We now support finetuning on the Qwen-VL models, including full-parameter finetuning, LoRA and Q-LoRA.

使用的qlora.sh
export CUDA_DEVICE_MAX_CONNECTIONS=1
DIR=pwd

MODEL="./Qwen-VL" # Qwen/Qwen-VL-Chat-Int4 Set the path if you do not want to load from huggingface directly

ATTENTION: specify the path to your training data, which should be a json file consisting of a list of conversations.

See the section for finetuning in README for more information.

DATA="data/demo_data.json"

export CUDA_VISIBLE_DEVICES=0

Remember to use --fp16 instead of --bf16 due to autogptq

python finetune.py
--model_name_or_path $MODEL
--data_path $DATA
--bf16 False
--fp16 True
--fix_vit True
--output_dir output_qwen
--num_train_epochs 5
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 8
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 1000
--save_total_limit 10
--learning_rate 1e-5
--weight_decay 0.1
--adam_beta2 0.95
--warmup_ratio 0.01
--lr_scheduler_type "cosine"
--logging_steps 1
--report_to "none"
--model_max_length 2048
--lazy_preprocess True
--gradient_checkpointing
--use_lora
--q_lora
--deepspeed finetune/ds_config_zero2.json

[xiayq1]

原来是要用量化的int4，不好意思。