swoook/transformers

Request a feature or examples to export BART for sequence classification to ONNX Runtime (ORT)

swoook opened this issue · 1 comments

swoook commented

🚀 Feature request

  • I'd like to export BART to ONNX Runtime
  • Of course, transformers officially supports exporting BART to ONNX Runtime [here]:

Ready-made configurations include the following models:

  • ALBERT
  • BART
  • BERT
  • CamemBERT
  • DistilBERT
  • GPT Neo
  • LayoutLM
  • Longformer
  • mBART
  • OpenAI GPT-2
  • RoBERTa
  • T5
  • XLM-RoBERTa
  • But it seems that it doesn't cover all of tasks:
  1. Exporting transformers models — transformers 4.12.5 documentation

The models showcased here are close to fully feature complete, but do lack some features that are currently in development. Namely, the ability to handle the past key values for decoder models is currently in the works.

  1. #13893 in huggingface/transformers (github)

    The generate method in ONNX isn't available yet. It is on the roadmap, but it is a big item that will require quite a bit of time to solve.

    You can follow the progress here: #13578

  • I'd like to implement a feature or manual to address this issue

Motivation

Your contribution

swoook commented