DeepVAC/deepvac

[BUG]CoreML模型转换双线差值上采样问题

MHGL opened this issue · 1 comments

MHGL commented

bug描述
在转化coreml过程中,如果代码中存在pytorch双线差值上采样,在ios13上会报错,在ios14上可以顺利执行。

如何复现
复现步骤:

# -*- coding:utf-8 -*-
import torch
from torch.nn import functional as F


class MyModule(torch.nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        ...

    def forward(self, x):
        return F.interpolate(x, scale_factor=2, mode="bilinear", align_corners=True)


if __name__ == "__main__":
    import coremltools

    net = MyModule()
    sample = torch.randn(1, 3, 288, 288)
    net = torch.jit.trace(net, sample).eval()

    sample = coremltools.TensorType(name="input", shape=(1, 3, 288, 288))
    coreml_model = coremltools.convert(model=net, inputs=[sample], minimum_deployment_target=coremltools.target.iOS13)

错误信息

WARNING:root:scikit-learn version 0.22.2.post1 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.
2021-08-11 16:39:44.386794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:root:TensorFlow version 2.4.1 detected. Last version known to be fully compatible is 2.3.1 .
WARNING:root:Keras version 2.4.3 detected. Last version known to be fully compatible of Keras is 2.2.4 .
Converting Frontend ==> MIL Ops:  83%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊                              | 5/6 [00:00<00:00, 4505.16 ops/s]
Running MIL optimization passes: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 16908.73 passes/s]
Translating MIL ==> MLModel Ops: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 20213.51 ops/s]
Traceback (most recent call last):
  File "converter.py", line 23, in <module>
    coreml_model = coremltools.convert(model=net, inputs=[sample], minimum_deployment_target=coremltools.target.iOS13)
  File "/home/liyang/.local/lib/python3.8/site-packages/coremltools/converters/_converters_entry.py", line 189, in convert
    check_deployment_compatibility(
  File "/home/liyang/.local/lib/python3.8/site-packages/coremltools/converters/mil/_deployment_compatibility.py", line 140, in check_deployment_compatibility
    raise ValueError(msg)
ValueError: Provided minimum deployment target requires model to be of version 4 but converted model uses following features which are available from version 5 onwards.
    1. Upsample operation with Align Corners mode

环境

  • 宿主机 cpu/ram/cuda设备: [比如. intel i5-9300H/8GB/GTX 1650 ]
  • 宿主机操作系统/内核版本/GPU驱动: [比如. ubuntu 20.04/5.4.0-80-generic/460.91.03
  • coremltools == 4.1
  • torch == 1.8.1
  • python == 3.8.10
MHGL commented
from coremltools.converters.mil import register_torch_op
from coremltools.converters.mil.frontend.torch.ops import _get_inputs, _get_scales_from_output_size
from coremltools.converters.mil.mil import Builder as mb

@register_torch_op(override=True)
def upsample_bilinear2d(context, node):
    inputs = _get_inputs(context, node)
    _input = inputs[0]
    output_size = inputs[1]
    align_corners = bool(inputs[2].val)

    if len(inputs) == 5:
        # For torch==1.5.0, upsample_bilinear2d has 5 inputs.
        scales_h = inputs[3]
        scales_w = inputs[4]

    scales = _get_scales_from_output_size(output_size, _input.shape)
    if scales:
        scales_h, scales_w = scales

    upsample_bilinear = mb.resize_bilinear(
        x=_input,
        target_size_height=output_size.val[0],
        target_size_width=output_size.val[1],
        name=node.name,
        sampling_mode="STRICT_ALIGN_CORNERS",
    )
    context.add(upsample_bilinear)


######################################################################
### line 12
######################################################################
h, w = x.shape[2:]
return F.interpolate(x, size=(2*h, 2*w), mode="bilinear", align_corners=True)