Had you tried Transformers's MSDA implementation?

[NineMeowICT]

I think Transformers library can provide a more general implementation, providing a better cross-platform capability.
I'm interested in it. If you had tried it, I wonder if it degraded the inference performance?

[NineMeowICT]

Sorry, I misunderstanded your model's architecture