allenai/OLMoE

Issues

• Early loss divergence for upcycling

  #15 opened 2 months ago by yazdayy
  5

• Dropout Regularization in expert modules

  #14 opened a month ago by taehyunzzz
  3

• Config Naming

  #16 opened a month ago by mchorton
  1

• Supported generative tasks

  #13 opened 2 months ago by taehyunzzz
  1

• recommended conf

  #12 opened 2 months ago by raingart
  1

• How to get the MMLU results in Table 4?

  #11 opened 2 months ago by mathfinder
  1

• Implementing MoE Sparse Upcycling

  #9 opened 2 months ago by adumans
  13

• llama.cpp / GGUF support

  #7 opened 3 months ago by sammcj
  7

• Tokenized dataset?

  #10 opened 2 months ago by joelburget
  1

• MOE Export Parallelism Training Script

  #8 opened 2 months ago by wdlctc
  5