Pinned issues
Issues
- 0
Load existing LORA and continue training it
#1887 opened by Nero10578 - 0
Jamba-1.5-Mini fine-tuning got ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.bfloat16
#1889 opened by coranholmes - 2
Preprocess --debug does not show newline \n token if previous string is ">" but shows if I add any other letter in the role fastchat
#1890 opened by Nero10578 - 0
Training with a large json dataset (>650K) throw error:pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays
#1888 opened by bofei5675 - 0
Support LGAI-EXAONE model
#1816 opened by shing100 - 1
Unable to load ORPO dataset in a *.json file
#1868 opened by SicariusSicariiStuff - 0
MixLoRA finetuning
#1880 opened by winglian - 3
accelerate launch bug on runpod
#1860 opened by JaynouOliver - 4
autoawq 0.2.6 causing conflicting dependencies
#1819 opened by Mr-Jeffery - 2
- 0
- 12
New changes in PEFT breaks FSDP (RLHF)
#1836 opened by maziyarpanahi - 1
Model saving issue after training
#1842 opened by gothaleshubham - 5
- 3
Mistral Nemo 12B training CUDA Out of memory only when enabling EVAL. On 2x3090Ti FSDP.
#1813 opened by Nero10578 - 0
inst chat jinja template does not match prompt format used while training with `conversation: mistral`
#1832 opened by nyxkrage - 0
Edge case for local dataset loading if its a folder
#1830 opened by ccdv-ai - 1
bitsandbytes==0.43.3 can't be installed on mac
#1826 opened by happylolonly - 1
- 4
Potential max_input_len Issue/Inconsistency?
#1749 opened by williambarberjr - 1
Support BAdam Optimizer
#1714 opened by bratao - 6
TinyLlama pretrain fails, but SFT works -- CUDA error: an illegal memory access was encountered
#1753 opened by chromecast56 - 4
Possible Bug in Chat Template Preprocessing
#1761 opened by hammoudhasan - 0
- 3
Colab Notebook Example In The Repo Not Running
#1784 opened by rajaboja - 3
Error Building Extension 'cpu_adam'
#1794 opened by e-p-armstrong - 2
Zamba2AttentionDecoderLayer.forward() takes from 4 to 10 positional arguments but 11 were given
#1799 opened by lucyknada - 2
`undefined symbol: cget_col_row_stats` when finetune in the axolotl-cloud docker container
#1802 opened by chengchengpei - 3
- 1
New Optimizer: Implement Adam-Mini optimizer
#1720 opened by SicariusSicariiStuff - 1
- 15
- 3
Data Gets Tokenized Before Special Tokens Are Added
#1770 opened by hammoudhasan - 0
- 0
- 0
increase tux scroll back buffer length in docker image
#1777 opened by winglian - 0
FSDP "zero-2" checkpoints don't save
#1776 opened by winglian - 4
Add a `chat_template` strategy for DPO datasets
#1708 opened by fozziethebeat - 1
Migrate multipack to refactored flash attention
#1774 opened by casper-hansen - 1
Training Freeze after "Shuffle merged datasets" (and adding position ids)
#1766 opened by e-p-armstrong - 2
Error unfreezing intermediate layers
#1747 opened by ayushsml - 7
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
#1750 opened by RishabhMaheshwary - 2
Support for Flash Attention 3
#1751 opened by creatorrr - 0
Spwaning multiple processes on card 0 when performing distributed training on a single machine with multiple cards?
#1741 opened by MengqingCao - 0
New Alignment Algorithm: SPPO
#1736 opened by kaykyr - 1
[Docs] Documentation Page 404
#1729 opened by tjtanaa - 0
Default to Chat Template in tokenizer_config
#1728 opened by hammoudhasan - 1
Gemma2 Support
#1719 opened by hammoudhasan - 2
Support loading a local hf dataset with `load_dataset`
#1715 opened by ccdv-ai - 0