Does the new flash-attention support ROCm?

[JiahuaZhao]

We refer to https://github.com/ROCm/flash-attention to install flash_attn with ROCm support (the current highest version is 2.0.4). When we need to do long context inference (using LongLoRA), sometimes errors occur: need flash-attn version ≥2.1.0. So wondering if there is a higher version of flash_attn that supports ROCm.

[tridao]

Sorry idk much about the ROCm version, you can ask on their repo.