GenAIModelExporter Component - parameter mismatch
samuel100 opened this issue · 3 comments
samuel100 commented
Describe the bug
In olive\passes\onnx\genai_model_exporter.py
the invocation of the create_model
method is not passing the right parameter names expected by builder.py
in the GenAI API:
To Reproduce
from olive.passes import GenAIModelExporter
from olive.hardware.accelerator import DEFAULT_CPU_ACCELERATOR
from olive.model import PyTorchModelHandler
from olive.model.config import HfConfig
# input a model from HF...
hf_cfg = HfConfig(
model_name="microsoft/phi-2",
task="text-generation"
)
input_model = PyTorchModelHandler(hf_config=hf_cfg)
x = GenAIModelExporter(
accelerator_spec=DEFAULT_CPU_ACCELERATOR,
config={"precision": "int4"},
)
x.run(
model=input_model,
data_root=".",
output_model_path="./models"
)
Other information
- OS: Windows 11
- Olive version: 0.5.0
- ONNXRuntime package and version: onnxruntime-genai==0.1.0rc4, onnxruntime-directml==1.17.1
jambayk commented
PR has been merged. Please try with olive installed from source. Thanks!
jambayk commented
closing since issue is resolved. please reopen if it still persists for you.