JohnSnowLabs/spark-nlp

Import Whisper large v3 into Spark NLP

minhduc210196 opened this issue · 5 comments

minhduc210196 commented

Link to the documentation pages (if available)

I use instructions at the link

#5669

for whisper loud v3 but got the following error. Please help me solve it
1231

How could the documentation be improved?

Please help me solve it

minhduc210196 commented

from sparknlp.annotator import *

whisper = (
WhisperForCTC.loadSavedModel(f"/content/export_onnx/openai/whisper-large", spark)
.setInputCols("audio_assembler")
.setOutputCol("text")

IllegalArgumentException: Size cannot be greater than Integer max value: 2547138560

maziyarpanahi commented

Hi @DevinTDHa
Is this a limitation on Java or we hard-coded a dimension somewhere?

DevinTDHa commented

Hmm this is a very curious error, I don't think anything is hard coded that would cause this. I will update this thread, once I have found the underlying issue!

  • ❤️1
DevinTDHa commented

Hi @minhduc210196,

I am trying to reproduce it, but I don't get quite the same error. It also fails on my side, because the large model is split into a graph file (*model.onnx) and a file with the weights (*model.onnx_data).

If it differs from the default notebook, could you share your environment, so I could try to reproduce it? Namely the version of transformers and optimum. Perhaps you could share a file with the output of pip freeze > pip_env.txt. Thanks!

PS: I was able to reproduce it, by renaming the encoder.onnx_data file to encoder.onnx. Is this what you did?

maziyarpanahi commented

It seems ONNX models above 2G will be exported into *model.onnx and *model.onnx_data files. Similar to other models we have, we should also adapt Whisper to use that feature since it can go above 2G and not assume there will be 1 file. (we support model.onnx_data)

  • 👍1