mosaicml/llm-foundry

Issues

• LLaMA PRO training resume problem

  #1231 opened 21 days ago by germanjke
  1

• Conversion Sharded -> Monolithic checkpoint

  #1220 opened a month ago by pretidav
  1

• Finetuning does not work on nightly

  #1221 opened a month ago by eldarkurtic
  1

• Add State Space Models / Mamba Layer Support

  #1174 opened a month ago by devin-ai-integration
  0

• Train with attention mask

  #1183 opened a month ago by germanjke
  1

• Issue when installing "pip install -e ".[gpu-flash2]""

  #955 opened 4 months ago by phutoan31299
  3

• MoE with FSDP

  #1197 opened a month ago by Muennighoff
  1

• Any plan for supporting DPO?

  #846 opened 5 months ago by lorabit110
  1

• Observing 1/2 the throughput on AMD MI250

  #1153 opened a month ago by staghado
  4

• Possibility of training with hostname instead IP

  #1180 opened a month ago by germanjke
  1

• Evaluation for long_context_tasks failed with a KeyError: 'continuation_indices'

  #1073 opened a month ago by songkq
  3

• Is there a way to figure out what dependencies are installed in the docker image?

  #1116 opened a month ago by sc-gr
  1

• Opt-3b Pretrain YAML config failing with mosaicml/llm-foundry/2.2.1_cu121_flash2-4aef5de docker

  #1141 opened 2 months ago by bhavnicksm
  1

• Can you add the pre-training of dbrx?

  #1074 opened 2 months ago by win10ogod
  3

• Fine-tune dbrx-instruct on a single VM with 8 H100s

  #1105 opened 2 months ago by hosseinsarshar
  1

• Can't create environment on A100 server

  #863 opened 2 months ago by eldarkurtic
  2

• convert_dataset_hf.py example stuck

  #906 opened 2 months ago by eldarkurtic
  2

• How does packing work for non-MPT models?

  #839 opened 2 months ago by lorabit110
  10

• Fly tokenization with multiple streams

  #1030 opened 2 months ago by germanjke
  12

• Installation issue from habana_alpha branch

  #1090 opened 2 months ago by palash04
  2

• Setting Dropout in MPT Prefix-LM after Exporting to HuggingFace Crashes during Fine-tuning

  #1046 opened 3 months ago by timsteuer
  2

• Composer crashes when attempting to load sharded checkpoint

  #998 opened 4 months ago by growlix
  3

• Wrong number of samples for C4?

  #968 opened 4 months ago by eldarkurtic
  2

• Freeze when using cpu offload

  #879 opened 4 months ago by gywlssww
  2

• How to support multi-threaded parallel data preprocessing?

  #870 opened 5 months ago by YixinSong-e
  11

• `ValueError` when following finetuning `mpt-7b-arc-easy--gpu.yaml` example with different default batch size

  #947 opened 4 months ago by ouhenio
  2

• How to run inference/convert_composer_to_hf.py with MPT-1B model on Habana Gaudi 2, file formats do not match

  #917 opened 4 months ago by greg-serochi
  4

• Is flops calculation correct?

  #909 opened 4 months ago by lorabit110
  2

• FP8 not working

  #903 opened 5 months ago by prigoyal
  2

• Loss curve differences for pretraining

  #910 opened 5 months ago by maxidl
  1

• Issues in using FP8 for MPT baselines on H100

  #885 opened 5 months ago by prigoyal
  2

• Loss spikes and explode with mpt-1b model pretrain

  #759 opened 5 months ago by sagorbrur
  1

• sharded model format

  #840 opened 5 months ago by j-Gaow
  2

• save_num_checkpoints_to_keep in huggingface ckpt callback?

  #682 opened 5 months ago by germanjke
  2

• Triton attention patch from Mistral

  #860 opened 5 months ago by germanjke
  1

• How to record loss curve?

  #854 opened 5 months ago by YixinSong-e
  0

• Error loading JSON fine-tuning datasets

  #852 opened 5 months ago by lorabit110
  1

• How to support new ICL task types in own codebase

  #848 opened 5 months ago by sanjari-orb
  9

• buffer is too small for requested array

  #753 opened 6 months ago by MLlove0402
  2

• Request to Support AWS Inferentia2 for More Cost-Effective and Faster Inference in MPT

  #767 opened 6 months ago by anjiefang
  1

• Converted PrefixLM HF snapshot must enable cache for generation in config

  #780 opened 6 months ago by timsteuer
  2

• RuntimeError: Tensors must be CUDA and dense

  #777 opened 6 months ago by karandua2016
  6

• PrefixLM is loaded as CausalLM after HuggingFace export

  #739 opened 7 months ago by timsteuer
  7

• Do we have to use StreamingDataset for pretraining runs?

  #696 opened 7 months ago by eldarkurtic
  1

• flash attention 2 setup.py

  #722 opened 7 months ago by germanjke
  1

• `eval.py` hangs when config yaml's model hparams don't match model checkpoint hparams

  #755 opened 7 months ago by growlix
  0

• Converting a composer seq2seq t5 model throws an exception

  #754 opened 7 months ago by timsteuer
  3

• Bloom Tokenizer doen't work

  #726 opened 7 months ago by robi56
  4

• Benchmarking GLUE tasks for in-context learning

  #707 opened 7 months ago by ashim95
  2

• MPT models on the Hub not working with `transformers` main

  #703 opened 7 months ago by younesbelkada
  5