Add ONNX support to all transformers in Spark NLP
maziyarpanahi opened this issue · 2 comments
maziyarpanahi commented
Embeddings:
- AlbertEmbeddings
- BertEmbeddings
- BertSentenceEmbeddings
- CamemBertEmbeddings
- DeBertaEmbeddings
- DistilBertEmbeddings
- LongformerEmbeddings
- RoBertaEmbeddings
- RoBertaSentenceEmbeddings
- XlmRoBertaEmbeddings
- XlmRoBertaSentenceEmbeddings
-
XlnetEmbeddingsnot supported - E5Embeddings
- InstructorEmbeddings
- MPNetEmbeddings
Classifiers:
- AlbertForQuestionAnswering
- AlbertForSequenceClassification
- AlbertForTokenClassification
- BertForQuestionAnswering
- BertForSequenceClassification
- BertForTokenClassification
- BertForZeroShotClassification
- CamemBertForQuestionAnswering
- CamemBertForSequenceClassification
- CamemBertForTokenClassification
- DeBertaForQuestionAnswering
- DeBertaForSequenceClassification
- DeBertaForTokenClassification
- DistilBertForQuestionAnswering
- DistilBertForSequenceClassification
- DistilBertForTokenClassification
- DistilBertForZeroShotClassification
- LongformerForQuestionAnswering
- LongformerForSequenceClassification
- LongformerForTokenClassification
- RoBertaForQuestionAnswering
- RoBertaForSequenceClassification
- RoBertaForTokenClassification
- RoBertaForZeroShotClassification
- [ ]
[TapasForQuestionAnswering]not supported - XlmRoBertaForQuestionAnswering
- XlmRoBertaForSequenceClassification
- XlmRoBertaForTokenClassification
- XlmRoBertaForZeroShotClassification
- [ ]
[XlnetForSequenceClassification] not supported - [ ]
[XlnetForTokenClassification] not supported
Seq2Seq:
Speech To Text:
Image Classification:
minhduc210196 commented
Whisper large in SparkNlp (Message tensorflow.GraphDef exceeds maximum protobuf size of 2GB)
minhduc210196 commented
MODEL_NAME = "openai/whisper-large"
Message tensorflow.GraphDef exceeds maximum protobuf size of 2GB: 2549145890