microsoft/Olive

GenAIModelExporter Component - parameter mismatch

samuel100 opened this issue · 3 comments

Describe the bug
In olive\passes\onnx\genai_model_exporter.py the invocation of the create_model method is not passing the right parameter names expected by builder.py in the GenAI API:

image

To Reproduce

from olive.passes import GenAIModelExporter
from olive.hardware.accelerator import DEFAULT_CPU_ACCELERATOR
from olive.model import PyTorchModelHandler
from olive.model.config import HfConfig

# input a model from HF...
hf_cfg = HfConfig(
    model_name="microsoft/phi-2",
    task="text-generation"
)

input_model = PyTorchModelHandler(hf_config=hf_cfg)


x = GenAIModelExporter(
    accelerator_spec=DEFAULT_CPU_ACCELERATOR,
    config={"precision": "int4"},
)

x.run(
    model=input_model,
    data_root=".",
    output_model_path="./models"
)

Other information

  • OS: Windows 11
  • Olive version: 0.5.0
  • ONNXRuntime package and version: onnxruntime-genai==0.1.0rc4, onnxruntime-directml==1.17.1

Hi, thanks for bring this up. I created a PR to fix this #1046. Could you try it?

PR has been merged. Please try with olive installed from source. Thanks!

closing since issue is resolved. please reopen if it still persists for you.