facebookresearch/cc_net

Issues

• how to only compute the perplexity of each paragraph using your language model with local data?

  #54 opened a year ago by rongjingyue423
  1

• Running on local files

  #22 opened 4 years ago by sashavor
  4

• 503 Server Error: Service Unavailable for url

  #53 opened a year ago by yangyang0202
  1

• 从wet格式中提取文本

  #46 opened 2 years ago by wwfcnu
  2

• Whether CC_Net provides an existing monolingual corpus

  #52 opened a year ago by yangyang0202
  0

• requests.exceptions.HTTPError: 503 Server Error: Service Unavailable for url

  #49 opened 2 years ago by Hieunohair
  1

• Can reproduce still run normally?

  #51 opened a year ago by newbietuan
  0

• Numerous Errors

  #45 opened 2 years ago by conceptofmind
  2

• win10 use cc_net

  #50 opened a year ago by z-x-x136
  0

• Error: Job not requeued because: timed-out and not checkpointable.

  #19 opened 4 years ago by hadifar
  12

• CC-100 in statmt version is different from paper

  #48 opened 2 years ago by nbqu
  0

• Annotation statistics

  #47 opened 2 years ago by mauriceweber
  0

• The final json files are not as expected

  #44 opened 2 years ago by nengyinyibeiwu
  0

• The questions about the stats json configuration file

  #42 opened 2 years ago by QHPHBias
  0

• EOFError: Compressed file ended before the end-of-stream marker was reached

  #11 opened 4 years ago by zl827154659
  4

• Inquiries about utilizing 2022 collected common rawl snapshots

  #40 opened 2 years ago by hyunmokky
  0

• Inquiries about korean datasets utilized in the CCNet pipeline

  #39 opened 2 years ago by hyunmokky
  1

• when use odoo 16.0 in pycharm show this Error

  #37 opened 2 years ago by mohamedGaber93
  0

• ModuleNotFoundError: No module named 'typing_extensions'

  #3 opened 5 years ago by alexandremuzio
  5

• Batch job submission failed: Invalid job array specification

  #31 opened 3 years ago by swgu98
  3

• 403 forbidden while downloading

  #35 opened 2 years ago by Raven-Ren
  2

• Error when Running 2020-34 dumps

  #16 opened 4 years ago by Phil1108
  4

• Error: Mining phase failure

  #36 opened 2 years ago by AssisRaphael
  1

• Variance of hash files sizes in newer crawls

  #27 opened 4 years ago by var926
  1

• requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url

  #14 opened 4 years ago by bioothod
  5

• sbatch: error: Batch job submission failed: Invalid job array specification

  #32 opened 3 years ago by swgu98
  0

• I want to copy the output data of CC_net directly, what should I do?

  #30 opened 3 years ago by mome1024
  1

• make dl_all_lm failing

  #20 opened 4 years ago by sashavor
  2

• cc_net/tools/dl_cc_100.py fails to extract complete dataset

  #25 opened 4 years ago by leezu
  6

• "Reproducing our work" does not specify set of languages and snapshots

  #26 opened 4 years ago by leezu
  2

• Question about the size of Roberta-small

  #28 opened 4 years ago by MatthewCYM
  0

• getpy version specified in setup.py no longer available

  #24 opened 4 years ago by leezu
  1

• support of Hausa

  #9 opened 4 years ago by donglixp
  4

• Model finding

  #21 opened 4 years ago by sashavor
  0

• Are not all languages in the paper supported?

  #18 opened 4 years ago by feddybear
  1

• Decrease RAM usage, investigate miss documents

  #6 opened 4 years ago by gwenzek
  3

• Cannot download the precpomputed files

  #7 opened 4 years ago by yinfeiy-g
  7

• Doing hashing, mining and regroup from each bin order

  #15 opened 4 years ago by aswin-giridhar
  1

• Any plans to release the cleaned datasets ?

  #1 opened 5 years ago by rohit-gupta
  6

• ERROR: Package u'cc-net' requires a different Python: 2.7.12 not in '>=3.7'

  #12 opened 4 years ago by Nanamumuhan
  2

• Failing to use mp execution

  #4 opened 5 years ago by alexandremuzio
  4

• Early exit when desired number of documents is reached?

  #5 opened 5 years ago by JohnGiorgi
  3

• Dedup all paragraphs if it appear more than once?

  #8 opened 5 years ago by xingenju
  2

• ChunkedEncodingError & ConnectionResetError

  #2 opened 5 years ago by soloice
  13