dola可以适配xverse模型吗？
Closed this issue · 1 comments
xxm1668 commented
dola可以适配xverse模型吗？
voidism commented
We have merged the DoLa decoding into the new version (4.39.0.dev0) of transformers package.
Install it here: https://github.com/voidism/transformers-dola
Follow the instructions here for decoding: https://github.com/voidism/transformers-dola/blob/main/docs/source/en/generation_strategies.md#dola-decoding
This implementation is agnostic to the model architecture, so it should support new models in huggingface.