axolotl-ai-cloud/axolotl

Issues

• Load existing LORA and continue training it

  #1887 opened 4 days ago by Nero10578
  0

• Jamba-1.5-Mini fine-tuning got ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.bfloat16

  #1889 opened 3 days ago by coranholmes
  0

• Preprocess --debug does not show newline \n token if previous string is ">" but shows if I add any other letter in the role fastchat

  #1890 opened 3 days ago by Nero10578
  2

• Training with a large json dataset (>650K) throw error:pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays

  #1888 opened 3 days ago by bofei5675
  0

• Support LGAI-EXAONE model

  #1816 opened a month ago by shing100
  0

• Unable to load ORPO dataset in a *.json file

  #1868 opened 11 days ago by SicariusSicariiStuff
  1

• MixLoRA finetuning

  #1880 opened 8 days ago by winglian
  0

• accelerate launch bug on runpod

  #1860 opened 11 days ago by JaynouOliver
  3

• autoawq 0.2.6 causing conflicting dependencies

  #1819 opened a month ago by Mr-Jeffery
  4

• ORPO results in `Cannot flatten integer dtype tensors`

  #1838 opened 17 days ago by maziyarpanahi
  2

• Pass `trust_remote_code` to `load_dataset(...)` for `datasets>=2.20.0`

  #1829 opened 20 days ago by ccdv-ai
  0

• New changes in PEFT breaks FSDP (RLHF)

  #1836 opened 15 days ago by maziyarpanahi
  12

• Model saving issue after training

  #1842 opened 16 days ago by gothaleshubham
  1

• I get ValueError when trying to run Axoltlt on pretraining dataset

  #1713 opened 15 days ago by Ahmedn1
  5

• Mistral Nemo 12B training CUDA Out of memory only when enabling EVAL. On 2x3090Ti FSDP.

  #1813 opened a month ago by Nero10578
  3

• inst chat jinja template does not match prompt format used while training with `conversation: mistral`

  #1832 opened 19 days ago by nyxkrage
  0

• Edge case for local dataset loading if its a folder

  #1830 opened 20 days ago by ccdv-ai
  0

• bitsandbytes==0.43.3 can't be installed on mac

  #1826 opened 21 days ago by happylolonly
  1

• A wrapper of RESTful API - Web server on the top of Axolotl

  #1807 opened a month ago by ulhaqi12
  1

• Potential max_input_len Issue/Inconsistency?

  #1749 opened 2 months ago by williambarberjr
  4

• Support BAdam Optimizer

  #1714 opened 2 months ago by bratao
  1

• TinyLlama pretrain fails, but SFT works -- CUDA error: an illegal memory access was encountered

  #1753 opened 2 months ago by chromecast56
  6

• Possible Bug in Chat Template Preprocessing

  #1761 opened a month ago by hammoudhasan
  4

• There is no model saved in the checkpoint/final when lora_tuning llama3

  #1804 opened a month ago by leoozy
  0

• Colab Notebook Example In The Repo Not Running

  #1784 opened a month ago by rajaboja
  3

• Error Building Extension 'cpu_adam'

  #1794 opened a month ago by e-p-armstrong
  3

• Zamba2AttentionDecoderLayer.forward() takes from 4 to 10 positional arguments but 11 were given

  #1799 opened a month ago by lucyknada
  2

• `undefined symbol: cget_col_row_stats` when finetune in the axolotl-cloud docker container

  #1802 opened a month ago by chengchengpei
  2

• Can't preprocess dataset using meta-llama/Meta-Llama-3.1-8B model

  #1800 opened a month ago by ohmeow
  3

• New Optimizer: Implement Adam-Mini optimizer

  #1720 opened 2 months ago by SicariusSicariiStuff
  1

• Training early stop without setting the early_stopping_patience parameters

  #1775 opened 2 months ago by leoozy
  1

• OOM Training 70B on 4x3090 24GB both FSDP or Deepspeed Zero3

  #1717 opened 2 months ago by Nero10578
  15

• Data Gets Tokenized Before Special Tokens Are Added

  #1770 opened a month ago by hammoudhasan
  3

• unhandled prompt tokenization strategy: sharegpt.load_glaive

  #1787 opened a month ago by hasan9090
  0

• deprecate dataset_processes in preference of dataset_num_proc

  #1783 opened a month ago by winglian
  0

• increase tux scroll back buffer length in docker image

  #1777 opened a month ago by winglian
  0

• FSDP "zero-2" checkpoints don't save

  #1776 opened a month ago by winglian
  0

• Add a `chat_template` strategy for DPO datasets

  #1708 opened 2 months ago by fozziethebeat
  4

• Migrate multipack to refactored flash attention

  #1774 opened 2 months ago by casper-hansen
  1

• Training Freeze after "Shuffle merged datasets" (and adding position ids)

  #1766 opened 2 months ago by e-p-armstrong
  1

• Error unfreezing intermediate layers

  #1747 opened 2 months ago by ayushsml
  2

• RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

  #1750 opened 2 months ago by RishabhMaheshwary
  7

• Support for Flash Attention 3

  #1751 opened 2 months ago by creatorrr
  2

• Spwaning multiple processes on card 0 when performing distributed training on a single machine with multiple cards？

  #1741 opened 2 months ago by MengqingCao
  0

• New Alignment Algorithm: SPPO

  #1736 opened 2 months ago by kaykyr
  0

• [Docs] Documentation Page 404

  #1729 opened 2 months ago by tjtanaa
  1

• Default to Chat Template in tokenizer_config

  #1728 opened 2 months ago by hammoudhasan
  0

• Gemma2 Support

  #1719 opened 2 months ago by hammoudhasan
  1

• Support loading a local hf dataset with `load_dataset`

  #1715 opened 2 months ago by ccdv-ai
  2

• Add Documentation on how to perform Multi-Node Finetuning in Slurm

  #1709 opened 3 months ago by juvi21
  0