jina-ai/jerboa

Issues

• Create correct outputs from Falcon by changing the generation configuration

  #103 opened a year ago by sebastian-weisshaar
  1

• Add support for deep speed

  #119 opened a year ago by alaeddine-13
  1

• Incorporate baize data and training into jerboa training pipeline

  #45 opened a year ago by alaeddine-13
  0

• Add automatic evaluation with gpt3

  #97 opened a year ago by alaeddine-13
  0

• Create an 8bits model for inference out of falcon 40b

  #68 opened a year ago by samsja
  0

• support codegen 1B in our training

  #120 opened a year ago by alaeddine-13
  0

• Python QA instruction tuning dataset

  #77 opened a year ago by JohannesMessner
  0

• Update HF models

  #93 opened a year ago by samsja
  1

• Create evaluation harness with ChatGPT

  #118 opened a year ago by sebastian-weisshaar
  0

• Support mosaicml dolly_hhrlhf dataset

  #114 opened a year ago by alaeddine-13
  0

• Align falcon 40b on code alpaca

  #32 opened a year ago by samsja
  1

• fixing transformers version

  #87 opened a year ago by samsja
  0

• add redpajama 7b to our pipeline

  #48 opened a year ago by samsja
  1

• Align Falcon 7b on Lima

  #80 opened a year ago by sebastian-weisshaar
  2

• Align falcon 7b on alpaca

  #34 opened a year ago by samsja
  0

• In our evaluation code we found a bug where the max token is just 128.

  #74 opened a year ago by alaeddine-13
  0

• There are cases where the model is not stopping or repeats itself. We will try training for longer and see what happens

  #73 opened a year ago by alaeddine-13
  1

• Add dolly 15k instruction dataset

  #96 opened a year ago by alaeddine-13
  0

• save full weights and upload to hf not just adapters

  #99 opened a year ago by azayz
  0

• We need to train alpaca-lora on the same number of lora layers to be able to compare it to falcon 7B and understand the effect of changing from llama to falcon

  #71 opened a year ago by alaeddine-13
  0

• Align Falcon 7B on Lima

  #79 opened a year ago by sebastian-weisshaar
  0

• For Falcon, there are cases where the generation outputs an EOS token but does not stop

  #72 opened a year ago by alaeddine-13
  0

• Add lima dataset to the training pipeline

  #64 opened a year ago by alaeddine-13
  1

• Align Falcon 40b on alpaca-lora

  #85 opened a year ago by sebastian-weisshaar
  0

• Experiment with Lightning fabric, reproduce speed improvement from: https://lightning.ai/pages/community/finetuning-falcon-efficiently/

  #86 opened a year ago by sebastian-weisshaar
  0

• Compare code aligned model to current SOTA

  #70 opened a year ago by azayz
  0

• Fix bug in save_pretrained

  #59 opened a year ago by samsja
  0

• Add dockerfile to jerboa to run on runpod

  #53 opened a year ago by samsja
  0

• Add red pajamas instruct dataset to our pipeline

  #49 opened a year ago by alaeddine-13
  0

• Pipeline training dataset refactoring

  #51 opened a year ago by alaeddine-13
  0

• logs dataset in wandb

  #52 opened a year ago by samsja
  0

• Align llama7 on code alpaca

  #33 opened a year ago by samsja
  1

• Add qlora to our current codebase

  #22 opened a year ago by samsja
  1

• Allign llama 7b on alpaca with 4 bits

  #31 opened a year ago by samsja
  0

• Long term: create a good evaluation for QA code

  #35 opened a year ago by samsja
  8

• Publish alpaca lora 8 bits on our HF account

  #36 opened a year ago by samsja
  1

• Prepare our code base to be able to ft on code alpaca

  #38 opened a year ago by samsja
  0

• Fix evaluation OOM

  #12 opened a year ago by JohannesMessner
  0

• Paper False promise llm

  #21 opened a year ago by samsja
  0

• WandB: Upload artifacts

  #11 opened a year ago by JohannesMessner
  1

• WandB: Remove unwante loss chart

  #13 opened a year ago by JohannesMessner
  1

• Create a tiny LLama model to run test

  #1 opened a year ago by samsja
  0