/gpt4all-experiments

Repo to train / inference different models with GPT4ALL dataset.

Primary LanguagePythonApache License 2.0Apache-2.0

gpt4all-experiments

Repo to train / inference different models with GPT4ALL dataset.

This repo is very much a WIP

Currently the following are available:

  • hacked together training code for GPT-J and potentially GPT-NeoX
  • inference code for GPT-J
  • Model LoRA weights available here: M-Chimiste/GPT-J-6B-GPT4ALL-SAMP-LORA-v1 (Current model is only trained on 50k samples of the GPT4ALL dataset as the full dataset would have taken over 5 days to train on my 2x 3090s. Hopefully we can do this when I next go on vacation).