shaigue/pmi_masking

Issues

• Add support for RedPajama

  #33 opened a year ago
  0

• does having a primary key in the table actuall nessesary / helpful?

  #32 opened a year ago
  0

• extract specs on the remote environments from the logs

  #31 opened a year ago
  0

• Add support for word Level tokenization

  #30 opened a year ago
  0

• deal with wikipedia bug

  #29 opened a year ago
  1

• write a blog post

  #28 opened a year ago
  0

• Try to optimize `count_ngrams_in_batches`

  #27 opened a year ago
  0

• autotune batch sizes according to system spec

  #26 opened a year ago
  0

• run medium bookcorpus on linux system to verify that everything is working well, and recover logging, and analyze output

  #25 opened a year ago
  0

• try to disentangle the dataset loading from my code, so that anyone could provide it's own dataset.

  #24 opened a year ago
  0

• setup and run tests on a remote machine with linux

  #23 opened a year ago
  0

• add checkpoints to save intermiddiate results in case of unexpected shutdown

  #21 opened a year ago
  0

• add random sampling support?

  #20 opened a year ago
  0

• Code review for create_pmi_masking_vocab.py

  #19 opened a year ago
  0

• write a descriptive README.md

  #18 opened a year ago
  0

• Try to optimize `aggregate_ngram_counts`

  #17 opened a year ago
  2

• document the performance results

  #16 opened a year ago
  0

• remove redundent files from the repo

  #15 opened a year ago
  1

• integration with LLM training code

  #14 opened a year ago
  1

• Visualize the flow of the program. This can help people understand what is going on

  #13 opened a year ago
  0

• prunning & sampling strategies to handle large amounts of data

  #12 opened a year ago
  0

• create pmi masking vocabulary for RedPajama

  #11 opened a year ago
  0

• reproduce results on wiki+bookcorpus

  #10 opened a year ago
  1

• add remote logging/monitoring

  #9 opened a year ago
  0

• add progress bar for the different stages

  #8 opened a year ago
  0

• find a way to test this code on a linux before handing it out.

  #7 opened a year ago
  0

• add support for different datasets structures

  #6 opened a year ago
  0

• add description of logging messages to the README.md

  #5 opened a year ago
  0

• add description for the main enterence script to README.md

  #4 opened a year ago
  0

• figure out how to do code review

  #3 opened a year ago
  0

• main enterence script with all arguments from command line

  #2 opened a year ago
  0

• finer grain control over which dataset to load

  #1 opened a year ago
  0