bitsandbytes-foundation/bitsandbytes

Issues

• Error when saving FSDP weights with cpu_offload=True [rank1]: AttributeError: 'Params4bit' object has no attribute 'absmax'

  #1405 opened 2 months ago by PanagiotisFytas
  3

• RuntimeError: CUDA Setup failed despite GPU being available.

  #1434 opened 25 days ago by kaijun123
  5

• ROCm6.2 Error cublasLt Encountered Error!

  #1428 opened a month ago by githust66
  0

• ARM Runners December 2024

  #1390 opened 13 days ago by johnnynunez
  1

• It seems that the current version of bitsandbytes is not compatible with my CUDA 12.4 library.

  #1409 opened 2 months ago by Yukiiceeee
  1

• Release v44 not available for Mac

  #1378 opened 16 days ago by ACMCMC
  8

• FSDP2 integration: torch.chunks(Params4bit) not returning Params4bit subclass

  #1424 opened a month ago by mreso
  1

• RuntimeError: Internal: could not parse ModelProto from /home/smusleh/miniGPT-Med/llama-2-7b-chat-hf/tokenizer.model

  #1438 opened 18 days ago by smusleh
  1

• CUDA setup failed on linux CUDA12.3 with version 0.45.0

  #1440 opened 17 days ago by viki760
  0

• aarch64 whl in PyPi

  #1437 opened 18 days ago by drikster80
  1

• python -m bitsandbytes ·

  #1423 opened a month ago by gunnusravani
  1

• Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback): CUDA Setup failed despite GPU being available. Please run the following command to get more information: python -m bitsandbytes Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

  #1387 opened 3 months ago by smillpine
  1

• BnB load_in_8bit model violates attention mask

  #1433 opened 25 days ago by poedator
  0

• bitsandbytes-0.44.1.dev0-py3-none-macosx_13_1_arm64.whl is not a supported wheel on this platform.

  #1406 opened 2 months ago by wuhongsheng
  1

• CUDA Setup failed despite GPU being available

  #1394 opened 2 months ago by carlxc911
  4

• CUDA Setup failed despite CUDA being available.

  #1404 opened 2 months ago by N1ana7mi
  2

• Rewrite `assert`s as exceptions

  #1408 opened 2 months ago by akx
  0

• Support for quantization of convolutional layers

  #1414 opened a month ago by JohnnyRacer
  0

• 8-bit C-Optim optimizers

  #1430 opened a month ago by odusseys
  0

• An error occurred: CUDA is required but not available for bitsandbytes.

  #1384 opened 3 months ago by GaoDalie
  1

• 'NoneType' object has no attribute 'cget_col_row_stats'

  #1427 opened a month ago by githust66
  1

• Source Code for `cadam_8bit_blockwise_grad_bf16`

  #1417 opened a month ago by ionutmodo
  1

• Anyway to change the quant_storage after the model is loaded?

  #1426 opened a month ago by AaronZLT
  0

• Installed with cuda support but doesn't detect gpu runtime

  #1421 opened a month ago by mahmoodn
  4

• Integration of BitNet into bitsandbytes

  #1412 opened a month ago by ParagEkbote
  1

• Device_map='auto' not working along with bitsandbytes (transformers)

  #1411 opened a month ago by guillemram97
  3

• [Windows] [Error] Compile from source

  #1398 opened a month ago by blap
  1

• Support running on CPU

  #1402 opened 2 months ago by fzyzcjy
  2

• Paged optimizer resuming from checkpoint - attributeError: 'int' object has no attribute 'cpu'

  #1381 opened 3 months ago by shivam15s
  2

• [Question] How to manually quantize every linear of a given model?

  #1415 opened a month ago by AaronZLT
  0

• Support

  #1407 opened 2 months ago by Qarqor5555555
  0

• unable to run FSDP2 with low bit optimizers like adam 8 bit

  #1403 opened 2 months ago by nighting0le01
  0

• Available version for 4-bit quantization supporting CUDA 11.1?

  #1388 opened 2 months ago by justin4ai
  0

• Questions about the details of LLM.int8

  #1400 opened 2 months ago by bg51717
  5

• too large numeric difference with pytorch inference

  #1396 opened 2 months ago by weixsong
  0

• I ran a NF4 72B model in 2xA6000 using llamafactory

  #1392 opened 2 months ago by charleswg
  0

• CUDA Architecture 80+ Causing Incorrect Model Behavior with BitsAndBytes Quantization

  #1391 opened 2 months ago by gunjunlee
  0

• AdEMA NaN when loading from state_dict

  #1382 opened 3 months ago by darius-lam
  1

• Python 3.9 support broken in 0.44.0

  #1376 opened 3 months ago by Benzhaomin
  0

• Model architecture is modified when I use BitsAndBytesConfig with default params

  #1371 opened 3 months ago by yunhao-tech
  0

• Release 0.44.0 does not belong to the current repository

  #1370 opened 3 months ago by ccoulombe
  2

• cuda is available but import bnb error

  #1355 opened 3 months ago by ZeroneBo
  2

• Merge LoRA into 405B

  #1359 opened 3 months ago by junzhang-zj
  4

• RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasGemmEx

  #1363 opened 3 months ago by LukeLIN-web
  1

• Bug when using optimizer LAMB 32bits

  #1350 opened 4 months ago by FrsECM
  0

• Lion Optimizer With Triton Kernel

  #1356 opened 3 months ago by lapp0
  1

• Model not able to quantize

  #1354 opened 3 months ago by alielfilali01
  0

• Torch autograd support for dequantize methods

  #1347 opened 4 months ago by yaldashbz
  0

• Cannot load decoder.lm_head.weight when loading 4 bit quantized model using VisionEncoderDecoder.from_pretrained

  #1343 opened 4 months ago by AditiJain14
  1

• quantize_4bit/dequantize_4bit gives wrong output on in-contiguous tensor

  #1342 opened 4 months ago by chenqianfzh
  0