deepsparse.TextGeneration doesn't accept `trust_remote_code` as an arg anymore
mgoin opened this issue · 1 comments
mgoin commented
The legacy text-generation pipeline supported trust_remote_code as an argument, which was inherited from the Transformers base pipeline here:
This was useful for models with custom code related to their tokenizer: #1176
Testing code:
from deepsparse import TextGeneration
pipeline = TextGeneration(model="hf:mgoin/llama2.c-stories15M-quant-ds", trust_remote_code=True)
print(pipeline("hello").generations[0].text)Output:
Fetching 7 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 28042.15it/s]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 724, in text_generation_pipeline
return Pipeline.create("text_generation", **kwargs)
File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 148, in create
raise e
File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 140, in create
pipeline = Operator.create(task=task, **new_kwargs)
File "/home/mgoin/code/deepsparse/src/deepsparse/operators/operator.py", line 137, in create
return operator_constructor(**kwargs)
File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/pipeline.py", line 175, in __init__
single_engine_operator = NLEngineOperator(
File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/nl_engine_operator.py", line 132, in __init__
super().__init__(**kwargs)
TypeError: EngineOperator.__init__() got an unexpected keyword argument 'trust_remote_code'