是否可以提供模型权重的.bin格式模型?
WangRongsheng opened this issue · 3 comments
WangRongsheng commented
是否可以提供模型权重的.bin格式模型?
BUAADreamer commented
You can first download the model from hf to the local path and then use the following code to get state_dict from safetensors
or bin
and save to bin files:
from pathlib import Path
from safetensors.torch import load_file
import torch
end_str = 'safetensors'
path_list = sorted(Path(old_state_dict_id).glob(f"*.{end_str}"))
state_dict = dict()
for path in path_list:
if end_str == 'safetensors':
state_dict_ = load_file(path)
else:
state_dict_ = torch.load(path, map_location="cpu")
state_dict.update(state_dict_)
WangRongsheng commented
thanks
WangRongsheng commented
from pathlib import Path
from safetensors.torch import load_file
import torch
import os
# 定义输入输出文件夹路径
input_folder = Path("Chinese-LLaVA-Med-7B")
output_folder = Path("chinese_med_bin")
# 确保输出文件夹存在
output_folder.mkdir(parents=True, exist_ok=True)
# 查找所有 .safetensors 文件
safetensors_files = sorted(input_folder.glob("*.safetensors"))
# 遍历每个 .safetensors 文件并进行处理
for safetensors_file in safetensors_files:
# 加载 safetensors 文件内容
state_dict = load_file(safetensors_file)
# 构造输出文件路径
output_file = output_folder / (safetensors_file.stem + ".bin")
# 保存为 .bin 文件
torch.save(state_dict, output_file)
print(f"Converted {safetensors_file} to {output_file}")
print("All files have been converted successfully.")