facebookresearch/xformers
Hackable and optimized Transformers building blocks, supporting a composable construction.
PythonNOASSERTION
Pinned issues
Issues
- 2
- 1
[feat] add python3.12 support
#1016 opened by Mon-ius - 0
Sparse Implementation is not triggered
#1054 opened by ryanliu30 - 0
How to integrate xformers into modeling_bert.py
#1053 opened by jiahuanluo - 10
- 1
Out of Memory on pip install
#1051 opened by celsowm - 1
FileNotFoundError: [WinError 3] The system cannot find the specified path: 'C:\\xformers\\third_party\\flash-attention\\flash_attn'
#1048 opened by metahades18892 - 0
- 1
Request for tutorial on how to modify an attention processor into its xformers version
#1047 opened by JWargrave - 6
- 2
Error in installing xformers for google colab connected to a GPU runtime
#1049 opened by UsamaSaddiqu - 2
`flshattF@v2.3.6` is not supported because: max(query.shape[-1] != value.shape[-1]) > 256
#1027 opened by qianghuangwhu - 2
Builds for Pytorch 2.4 (CUDA 12.1)?
#1044 opened by chi0tzp - 1
Autocast does not work with xformers
#1046 opened by Luciennnnnnn - 1
- 0
suitable for L40
#1043 opened by Sariel2 - 3
- 1
ERROR: Could not build wheels for xformers, which is required to install pyproject.toml-based projects
#1023 opened by greasebig - 1
- 3
Built wheel for torch 2.3.0
#1034 opened by daxijiu - 3
AttributeError: 'str' object has no attribute 'impl'
#1035 opened by le-wei - 2
attention's flop calculation when casual is set to True.
#1033 opened by kf-zhang - 0
Masked MLP
#1038 opened by nicolas-dufour - 0
- 1
Xformers are slowing down on 3090/4090
#1031 opened by rikeLiu - 0
Compiling took more than 100GB memory in 4090
#1032 opened by henrywoo - 0
BlockDiagonalAttention computes NaN gradients, when using bfloat16 and deterministic torch
#1025 opened by nimia - 0
output from memory_efficient_attention not exactly the same with pytorch equivalent implementation
#1024 opened by wangh09 - 2
[Bug] benchmark shows 0.00MB consumed
#1019 opened by deciding - 2
- 3
- 0
swiglu_op _Stack_or_None RunTimeError
#1022 opened by yup3antarctica - 2
Install xformers without changing pytorch version
#1020 opened by yup3antarctica - 0
How to use Relative Position Bias with xformers
#1015 opened by CFOP-xyn - 2
Built wheel for torch 2.2.2
#1010 opened by daxijiu - 1
- 0
Xformers can not perform memory_efficient_attention
#1013 opened by axel578 - 0
Old version of Xformers fit for lower torch
#1008 opened by ccxi1008 - 0
MicroGPT example uses encoders
#1007 opened by rteehas - 0
fix link in examples README.md
#1006 opened by melvinebenezer - 7
- 2
Cant install xformers
#999 opened by OkGamez313 - 1
100 errors detected in the compilation of "/tmp/tmpxft_0000e855_00000000-6_attention_backward_generic.cpp1.ii"
#1004 opened by xh741 - 6
Library raises `NameError` if _has_cpp_library is False
#1000 opened by swails - 0
- 0
Cross-attention
#1001 opened by theophilegervet - 5
- 0
Support Taylor Series Linear Attention
#997 opened by hoangmit - 0
pip install from a specific branch
#994 opened by QiqLiang - 1
Does xformer require CUDA or can it also be run on TPU/Tranium instances?
#993 opened by TommasoBendinelli