EleutherAI/gpt-neox

Support for Mosaic Models

rajveer43 opened this issue · 1 comments

rajveer43 commented
Support for Mosaic Models
StellaAthena commented

Mosaic models use our tokenizer, so the main to-do items are:

  1. Are there any configs or features they use that we don't yet support?
  2. Converting the checkpoints to the GPT-NeoX format