sophgo/tpu-mlir

Rewrite Patterns for PermuteOp

charlesxzb opened this issue · 3 comments

charlesxzb commented

This task is divided into 5 subtasks, and a PR can be submitted after completing each task.

Note: ONNX Transpose will be converted as PermuteOp in TPU-MLIR.

Task1:

  1. Create a Trait named SupportPermuteMove, which is similar to SupportFuseRelu.
  2. Add SupportPermuteMove Trait to Top Relu, AddConst, Cast, Sigmiod in tpu-mlir/include/tpu_mlir/Dialect/Top/IR/TopOps.td.
  3. When there is transpose + op in the model, and the op has SupportPermuteMove, apply the pattern to convert transpose + op to op + transpose (i.e., move the transpose after the op).
  4. Create a unit test Transpose + (Relu, Addconst, Cast and Sigmiod) in test_onnx.py to test the pattern.
  • Code path: tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • Refer to SupportFuseRelu for creating SupportPermuteMove.

Task2:

  1. Create a Trait named SupportAxisPermuteMove, which is similar to SupportFuseRelu.
  2. Add SupportPermuteMove to Top Softmax and Topk in tpu-mlir/include/tpu_mlir/Dialect/Top/IR/TopOps.td.
  3. When there is transpose + op in the model, and the op has SupportAxisPermuteMove, apply the pattern to convert transpose + op to op + transpose, as well as modify the axis attribute.
  4. Create unit test Transpose + (Softmax and Topk) in test_onnx.py to test the pattern.
  • Code path: tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • Refer to SupportFuseRelu for creating SupportAxisPermuteMove.

Task3:

  1. Add a pattern that converts transpose + pad to pad + transpose, and modify the paddings attribute.
  2. Create a unit test Transpose + Pad in test_onnx.py to test the pattern.
  • Code path: tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp

Task4:

  1. Add a pattern that converts transpose + binary to binary + transpose when one input of binary is weight.
  2. Create a unit test Transpose + Binary (Add, Sub and Mul. One of the inputs is weight) in test_onnx.py to test the pattern.
  • Code path: tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • Note: The weight needs to be modified as well.

Task5:

  1. Add a pattern that converts transpose + binary to binary + transpose when the two inputs of binary are transposed, and the order of transpose is the same.
  2. Create a unit test Transpose + Binary (Add, Sub, Mul) in test_onnx.py to test the pattern.
  • Code path: tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp

About how to add a Top Operator pattern, please refer to How to Add a Top Operator Rewrite Pattern?

中文版------------------------------------------------------------------------------------------------

该任务分为5个子任务，每完成一个任务都可以提交一次PR。

注意：ONNX Transpose 将在 TPU-MLIR 中转换为 PermuteOp。

Task1:

  1. 新建SupportPermuteMove的Trait，类似于SupportFuseRelu；
  2. 在tpu-mlir/include/tpu_mlir/Dialect/Top/IR/TopOps.td中对Top层relu、addconst、cast、sigmiod添加SupportPermuteMove的Trait；
  3. 当网络结构中存在transpose + op，且该op SupportPermuteMove的情况时，添加pattern将transpose + op转换为op + transpose （即将Transpose算子后移）。
  4. 在test_onnx.py中分别建立单元测试transpose + (relu、addconst、cast、sigmiod)，测试该pattern功能。
  • 代码路径：tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • SupportPermuteMove可以参考SupportFuseRelu。

Task2:

  1. 新建SupportAxisPermuteMove的Trait，类似于SupportFuseRelu ;
  2. 在tpu-mlir/include/tpu_mlir/Dialect/Top/IR/TopOps.td中对Top层softmax、topk添加SupportPermuteMove；
  3. 当网络结构中存在transpose + op，且该op SupportAxisPermuteMove的情况时，添加pattern将transpose + op转换为op + transpose，且对axis属性进行转换。
  4. 在test_onnx.py中分别建立单元测试transpose + (softmax、topk)，测试该pattern功能。
  • 代码路径：tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • SupportAxisPermuteMove可以参考SupportFuseRelu。

Task3:

  1. 添加pattern，将transpose + pad转换为pad + transpose，且对paddings属性进行转换。
  2. 在test_onnx.py中建立单元测试transpose + pad，测试该pattern功能。
  • 代码路径：tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp

Task4:

  1. 添加pattern，当binary的一个输入为weight时，将transpose + binary转换为binary + transpose。
  2. 在test_onnx.py中建立单元测试transpose + binary (add、sub、mul。且一个输入为weight)，测试该pattern功能。
  • 代码路径：tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp
  • 注意事项：需要修改weight排列

Task5:

  1. 添加pattern，当binary的两个输入都包含transpose，且transpose的order一致时，将transpose + binary转换为binary + transpose。
  2. 在test_onnx.py中建立单元测试transpose + binary (add、sub、mul)，测试该pattern功能。
  • 代码路径：tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp

关于如何添加一个Top层算子的Pattern，可以参考如何添加一个Top层算子的Rewrite Pattern?

Learnmore666 commented

在完成Task1中发现，当更改pattern后，可以成功生成转换后的mlir文件，但这里的python测试程序会导致数据对比不通过，这主要是因为在进行数据对比时，onnx模型推理文件的生成于mlir文件pass优化之前，因此onnx模型的推理和mlir模型在的推理时并不相同，带来了测试的报错。

Silence-Zhang-beijng commented

在完成Task1中发现，当更改pattern后，可以成功生成转换后的mlir文件，但这里的python测试程序会导致数据对比不通过，这主要是因为在进行数据对比时，onnx模型推理文件的生成于mlir文件pass优化之前，因此onnx模型的推理和mlir模型在的推理时并不相同，带来了测试的报错。

的确会产生数据比对不过的情况。此时应设置新的loc name。此数据不进行比对。

Knight-9 commented

想咨询一下剩余两个task是否还存在，因为看到源代码tpu-mlir/lib/Dialect/Top/Canonicalize/Permute.cpp内容好像已经变更了很多