how to set the blocks parameters when merge 2 flux models
Opened this issue · 4 comments
hello，I'm coming. Sorry to bother you againThis time, I would like to ask how to set the blocks parameters when merge 2 flux models. i know a flux model have 19 double_blocks,38 single_blocks,and base blocks. then in blocks node,How to set the value of blocks.
0.5,0.7,0.3,0.6,0.2,0,0.7....... How many digits are there in total？ 58？
Also, have you considered developing a blocked flux lora node? I am currently conducting tests on blocked Lora. I think if it can be merged into a model by ignoring certain blocks, it may overlook some of its negative effects.
By the way, when I tried to merge a Lora into the model, it couldn't continue to run
ComfyUI Error Report
Error Details
- Node Type: Mecha Merger
- Exception Type: RuntimeError
- Exception Message: cannot load a non-lora network using the lora model type. (D:\sd重新安装时的临时备份\models\Lora\Flux\flux-面部彩绘女孩_面部彩绘图案1.0 触发词xinyu_tiehua.safetensors)
found key not matching allowed suffixes ('.lora_up.weight', '.lora_down.weight', '.alpha'): transformer.single_transformer_blocks.0.attn.to_k.lora_A.weight
Stack Trace
File "D:\ComfyUI\execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "D:\ComfyUI\execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "D:\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "D:\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "D:\ComfyUI\custom_nodes\comfy-mecha\nodes\merge_nodes.py", line 198, in execute
merger.merge_and_save(
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\recipe_merger.py", line 79, in merge_and_save
model_config = recipe.accept(DetermineConfigVisitor())
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\recipe_nodes.py", line 118, in accept
return visitor.visit_merge(self, *args, **kwargs)
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\model_detection.py", line 99, in visit_merge
configs = [
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\model_detection.py", line 100, in <listcomp>
model.accept(self)
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\recipe_nodes.py", line 51, in accept
return visitor.visit_model(self, *args, **kwargs)
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\model_detection.py", line 90, in visit_model
node.model_type.convert_header(state_dict_path, node.state_dict, node.model_arch),
File "D:\ComfyUI\python\lib\site-packages\sd_mecha\extensions\model_type.py", line 64, in convert_header
raise RuntimeError(
Flux loras have their own state dict keys, currently there is no code to match the keys to the model in mecha. We need to add a conversion function, and then it might be able to work.
In general, flux support is currently very limited. The base lib wasn't completely designed to easily support many types of model without duplicating a bit of code. I was hoping to ease this process in the upcoming v0.1.0 update, but the PR is starting to take the dust despite my best intentions. I think I need to sit down and design this from first principles, from the bottom up and then back down, or something like that. It appears to be more complex than I thought.
Also, have you considered developing a blocked flux lora node?
Yes, this is something I have been considering for a while. Many people say that the current node isn't super easy to use, and I agree. I just haven't gotten around to it yet, as I have came to be quite busy since early september. It's fairly straightforward to implement but you kind of have to know how to dynamically create nodes and how to interface the sd-mecha lib I guess.