dhbrojas/gopilot

Issues

• Setup `torch.distributed` for Multi-GPU Training

  #25 opened 2 years ago
  0

• Use the Adam8Bit Optimiser

  #23 opened 2 years ago
  0

• Use Meet-in-the-Middle Objective

  #22 opened 2 years ago
  0

• Optimize GPT-2 Model using MosaicML

  #21 opened 2 years ago
  0

• Look into FlashAttention

  #20 opened 2 years ago
  0

• Enable CUDA Mixed Precision Training

  #12 opened 2 years ago
  0

• Enhance Data Processing/Filtering using Tokeniser Output

  #11 opened 2 years ago
  0

• Enable training with DeepSpeed for optimised Transformer kernels

  #9 opened 2 years ago
  0

• Create a Tokenizer using the Go Scanner

  #8 opened 2 years ago
  0

• Upload The Stack Dedup v1.2 to S3

  #4 opened 2 years ago
  0

• Leverage Hugging Face GPTBigCode Model

  #3 opened 2 years ago
  0

• Switch to PyTorch 2.0 Nightly to leverage latest Triton integration

  #1 opened 2 years ago
  0