Thank You for Your Work, and Query About Code Warnings.

[MinGiSa]

First, I would like to express my gratitude for the excellent work.

I do have a question: When I run your code,
I encounter the following warning. Is xformers not being used?

.\layers\swiglu_ffn.py:51: UserWarning: xFormers is not available (SwiGLU)
warnings.warn("xFormers is not available (SwiGLU)")
.\layers\attention.py:28: UserWarning: xFormers is not available (Attention)
warnings.warn("xFormers is not available (Attention)")
.\layers\block.py:35: UserWarning: xFormers is not available (Block)
warnings.warn("xFormers is not available (Block)")

[guojiajeremy]

Yes. xFormers is not used. Just ignore it.