EleutherAI/gpt-neox

Issues

• 'intermediate_size' not set in tools/ckpts/convert_neox_to_hf.py for neox model architecture

  #1208 opened 13 days ago by jvendrow
  3

• My servers used for multi-node training do not have ssh. How can I launch multi-node training using the torchrun command?

  #1203 opened 25 days ago by dingning97
  1

• LoRA Support

  #1204 opened 25 days ago by Quentin-Anthony
  1

• FileNotFoundError thrown when training

  #1127 opened a month ago by obicons
  5

• NCCL error in: ProcessGroupNCCL.cpp:1269, internal error, NCCL version 2.14.3

  #1147 opened a month ago by mackmake
  4

• How to convert gpt-neox to llama architecture..?

  #1151 opened a month ago by yuri-son
  1

• is there any ignore_index ability in the loss calculation?

  #1193 opened a month ago by exnx
  2

• بهترین تعمیرگاه موبایل در مشهد مقدس

  #1173 opened a month ago by rezaarefi
  1

• Large model instantiation using `DeepSpeed.zero.Init` under ZeRO-3

  #1189 opened 2 months ago by R0n12
  1

• Is there a way to train on the entire dataset for N epochs without specifying train-iters?

  #1164 opened 2 months ago by javirandor
  4

• continue training from a checkpoint with different number of gpu/node

  #1158 opened 2 months ago by mackmake
  1

• Integrate TransformerEngine

  #1098 opened 5 months ago by Quentin-Anthony
  6

• Add basic Mamba block

  #1148 opened 2 months ago by Quentin-Anthony
  0

• MoE loss variable not defined in gpt j residual code path

  #1174 opened 2 months ago by tf-nv
  0

• pipe_parallel_size = 1 using DeepSpeed PipelineEngine

  #1172 opened 2 months ago by DayOfThePenguin
  1

• Converting Pythia checkpoint from HF to NeoX fails

  #1161 opened 2 months ago by malteos
  3

• Dockerfile installation fails to run pythia 14M

  #1165 opened 2 months ago by tf-nv
  2

• Add Basic RWKV Block to GPT-NeoX

  #1167 opened 3 months ago by Quentin-Anthony
  0

• [BUG] Setting Finetune=True causes checkpoint loading to not work correctly

  #1121 opened 3 months ago by exnx
  5

• ImportError: /media/h/nvme/gpt-neox/.venv/lib/python3.8/site-packages/flash_attn_2_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol:

  #1079 opened 6 months ago by Drzhivago264
  1

• PyTorch Lightning Fused optimizer step

  #1160 opened 3 months ago by jahatef
  0

• files in multi-node training

  #1146 opened 3 months ago by mackmake
  2

• Support Mistral Models

  #1050 opened 3 months ago by Quentin-Anthony
  8

• Argument List too long error

  #1076 opened 3 months ago by kavlekar101
  2

• Update to current versions of python and pytorch

  #1143 opened 3 months ago by segyges
  0

• Port NVIDIA Nsight profiling to gpt-neox

  #1134 opened 3 months ago by Quentin-Anthony
  0

• Add PyTorch Memory Profiler

  #1152 opened 3 months ago by Quentin-Anthony
  0

• Tests fail when run with pytest --forked

  #1132 opened 4 months ago by segyges
  1

• Support for custom model architecture

  #1117 opened 3 months ago by itsnamgyu
  9

• Convert HF format or raw weights of Llama2 to NEOX format

  #1112 opened 3 months ago by fmh1art
  0

• Add Instructions for Loading Llama2 Models

  #1051 opened 3 months ago by Quentin-Anthony
  5

• some Datasets are not available

  #1071 opened 5 months ago by vangogh0318
  2

• Error on inference of huggingface

  #1142 opened 3 months ago by mackmake
  2

• calculate epoch

  #1140 opened 3 months ago by mackmake
  4

• [BUG?] Higher "gradient_accumulation_steps" still increases memory usage a lot

  #1123 opened 4 months ago by exnx
  3

• Add a Contributor Guide

  #1110 opened 4 months ago by Quentin-Anthony
  0

• Create Singularity Container

  #1119 opened 4 months ago by Quentin-Anthony
  1

• convert_hf_to_module（pipeline_parallel>1）

  #1092 opened 4 months ago by liuxinxin123
  2

• Support for lm_eval 0.4.0

  #1114 opened 4 months ago by ZhiYuanZeng
  4

• Apply new fused rotary embedding

  #1077 opened 4 months ago by Quentin-Anthony
  10

• Help with: No such file or directory: '/fsx/hailey/math-lm/gpt-neox/megatron/fused_kernels'

  #1083 opened 5 months ago by andrewarrow
  5

• Error in FLOPS Calculation

  #1093 opened 5 months ago by passaglia
  0

• instruction finetune

  #1091 opened 5 months ago by liuxinxin123
  1

• Finetune

  #1088 opened 5 months ago by liuxinxin123
  1

• Port DeepSpeed Ulysses

  #1078 opened 6 months ago by Quentin-Anthony
  1

• Interoperability and GPT-NeoX

  #1058 opened 7 months ago by StellaAthena
  2

• Recent LR Scheduler change does not account for inference/evaluation

  #1059 opened 7 months ago by dashstander
  0

• Support for Mosaic Models

  #1057 opened 7 months ago by rajveer43
  1

• ImportError: cannot import name 'helpers' from 'megatron.data'

  #1045 opened 8 months ago by shaunstoltz
  0

• AssertionError: Not sure how to proceed, we were given deepspeed configs in the deepspeed arguments and deepspeed.initialize() function call

  #1043 opened 8 months ago by shaunstoltz
  0