TypeError: NoneType takes no arguments
lvjin521 opened this issue · 4 comments
I reported the following error while executing the startup script:
python main.py --config-path configs/config-dev-offload-1-4090.json --port 6006 --host 0.0.0.0
You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in huggingface/transformers#24565
Traceback (most recent call last):
File "/workspace/flux-fp8-api/main.py", line 199, in
main()
File "/workspace/flux-fp8-api/main.py", line 159, in main
app.state.model = FluxPipeline.load_pipeline_from_config_path(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/flux-fp8-api/flux_pipeline.py", line 679, in load_pipeline_from_config_path
return cls.load_pipeline_from_config(config, debug=debug)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/flux-fp8-api/flux_pipeline.py", line 703, in load_pipeline_from_config
flow_model = quantize_flow_transformer_and_dispatch_float8(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/flux-fp8-api/float8_quantize.py", line 490, in quantize_flow_transformer_and_dispatch_float8
swap_to_cublaslinear(flow_model)
File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/flux-fp8-api/float8_quantize.py", line 380, in swap_to_cublaslinear
cublas_lin = CublasLinear(
^^^^^^^^^^^^^
TypeError: NoneType takes no arguments
Ah yeah I'll look into this, shouldn't be happening. Is caused by the https://github.com/aredden/torch-cublas-hgemm library not being installed. Should be a quick fix. I'll do it today.
Wow, thank you so much for your optimization, I have no problem now, you are great