A question about the pretrained LLava weight?

[RongkunYang]

Dear authors.
Thanks for your good work.
When I try to prepare the LLaVA weight, I follow the instructions as the LLaVa repository,
run the command "python3 -m llava.model.apply_delta
--base /path/to/llama-13b
--target /output/path/to/LLaVA-13B-v0
--delta liuhaotian/LLaVA-13b-delta-v0".
This cannot formulate the llava weight, because the Tokenizer can not be instantiated.
I found that this is because the file tokenizer_config.json maps the special tokens such as "bos_token" and "eos_token" to " ". But I don't know whether it's OK to change this file.
Or there are other methods?
Thank you.

The error message is as following, the special tokens of " " seems to be trapped in an infinite loop:
"""
File "/home/yrk/Desktop/program/LLM/LLaVA/llava/model/apply_delta.py", line 48, in
apply_delta(args.base_model_path, args.target_model_path, args.delta_path)
File "/home/yrk/Desktop/program/LLM/LLaVA/llava/model/apply_delta.py", line 20, in apply_delta
delta_tokenizer = AutoTokenizer.from_pretrained(delta_path)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
return cls._from_pretrained(
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 126, in init
self.update_post_processor()
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 136, in update_post_processor
bos_token_id = self.bos_token_id
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1136, in bos_token_id
return self.convert_tokens_to_ids(self.bos_token)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/home/yrk/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
"""

[XuweiyiChen]

Current version of LLM-Grounder does not use LLaVA anymore.