EleutherAI/pythia

Issues

• how to use Pythia

  #160 opened a month ago by gaohang
  0

• Reshape error in batch viewer

  #158 opened a month ago by activatedgeek
  1

• Convert to GGUF

  #159 opened a month ago by yanxon
  0

• tokenizer.pad_token

  #156 opened 2 months ago by vincent317
  1

• instruct-tuned pythia

  #155 opened 3 months ago by WilliamsToTo
  0

• Inconsistent init methods of pythia-6.9b model

  #135 opened 7 months ago by mqyqlx
  2

• Provide the shuffled index_mapping npy files for ease of reproducing training data

  #153 opened 3 months ago by ziqi-zhang
  0

• Optimizer states in HF format

  #152 opened 3 months ago by seyuboglu
  1

• Weird inconsistency in Tokenizer vocabulary

  #151 opened 3 months ago by javirandor
  1

• release of checkpoints of different steps

  #101 opened a year ago by TobiasLee
  5

• Is there existing code to resume training from specific checkpoint?

  #150 opened 3 months ago by javirandor
  1

• "gas" configuration doesn't do anything

  #149 opened 4 months ago by segyges
  0

• Missing / undownloadable checkpoints on huggingface

  #141 opened 4 months ago by mirandrom
  3

• Reading data is slowly！

  #126 opened 7 months ago by Lisennlp
  1

• Would it be possible to share training loss curves on the original Pythia models?

  #145 opened 5 months ago by itsnamgyu
  4

• Mismatch about the evaluation results

  #118 opened 7 months ago by yuzc19
  11

• [Pythia on Pile-Dedup] Training for ~1.5 epochs: how to identify the repeated sequences (i.e., the additional .5 epoch)?

  #144 opened 5 months ago by pietrolesci
  3

• Batch Viewer : Why Sequence Length 2049?

  #123 opened 7 months ago by prakharg24
  11

• Has the data been shuffled?

  #127 opened 7 months ago by Lisennlp
  2

• Details about "EleutherAI/pythia-160m-seed*" models

  #142 opened 6 months ago by IanMagnusson
  3

• Pytia or GPT-neox?

  #138 opened 6 months ago by borgr
  1

• .

  #140 opened 6 months ago by ParthaKrPaul
  2

• Wrong files in eval?

  #139 opened 6 months ago by borgr
  0

• Replicating the Training Data Order

  #136 opened 7 months ago by prakharg24
  1

• Deduplicated Pile dataset with Domain Attribution

  #137 opened 7 months ago by michaelduan8
  0

• the loss of pythia training

  #97 opened a year ago by Wangpeiyi9979
  3

• The value of weight decay

  #132 opened 7 months ago by yehuitang
  1

• Error when running unshard_memmap.py

  #114 opened 7 months ago by ShaneeyS
  2

• Model Initialization Question

  #129 opened 7 months ago by yanlai00
  1

• The performance about pythia and LLaMA model architecture

  #122 opened 7 months ago by peiyingxin
  1

• Any results on the validation set?

  #121 opened 8 months ago by chujiezheng
  1

• Weights tying

  #117 opened 8 months ago by link-er
  1

• Convert the huggingface checkpoint to GPT-Neox checkpoint

  #116 opened 10 months ago by ZhiYuanZeng
  2

• Clarification of Pythia tokenizer(s) at different sizes, steps and data preprocessing?

  #115 opened 10 months ago by RylanSchaeffer
  1

• Difference between LFS and HuggingFace datasets?

  #112 opened a year ago by eric-mitchell
  1

• Is there a template poilerplate for the prompt used in C.1 gender bias intervention?

  #106 opened a year ago by ruyuan-zuo
  1

• Is there an access to the deduplicated version of the data with meta info?

  #92 opened a year ago by Jason3900
  6

• Can I provide custom data and continue training Pythia on this new data?

  #113 opened a year ago by GeorgiAngelov
  1

• Multiple training runs of same model with different random seed for weight initialisation

  #110 opened a year ago by KarolisRam
  1

• Possible error in Pythia-12B-deduped step 32000

  #108 opened a year ago by smahdavi4
  2

• pythia-12b checkpoints missing on HuggingFace for step4000 and step32000

  #107 opened a year ago by byungdoh
  2

• Revamp experiment organization and migrate code when necessary

  #99 opened a year ago by StellaAthena
  0

• Will memorization experimental codes be released?

  #98 opened a year ago by chujiezheng
  2

• Train/valid/test split

  #102 opened a year ago by choidami
  1

• Fine-tuning recommendations

  #96 opened a year ago by RainIwakura
  2

• Are pythia_v0 and the new pythia_v1 models using the same input embedding matrix?

  #85 opened a year ago by levmckinney
  3

• Pythia 6.9B Model Missing Checkpoint

  #94 opened a year ago by chujiezheng
  1

• Training time or approximation of TFLOPs?

  #87 opened a year ago by zetian1025
  2

• Fine tune for text generation on custom data

  #89 opened a year ago by samarthsarin
  1

• training logs

  #86 opened a year ago by stjaco
  1