glample/fastBPE

Issues

• fastBPE will not install on windows 11 or WSL

  #64 opened 7 months ago by Otto-V-Bismarck
  3

• clang: error: no such file or directory: 'fastBPE/fastBPE.cpp'

  #60 opened a year ago by ver007
  3

• terminate called after throwing an instance of 'c10::Error' what(): [enforce fail at inline_container.cc:319] . unexpected pos 2302963008 vs 2302962896 frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x55 (0x15158b1320a5 in /usr/local/lib/python3.8/dist-packages/torch/lib/libc10.so)

  #63 opened 9 months ago by lldd1001
  0

• Fails to build on Python 3.11 - longintrepr.h: No such file or directory

  #62 opened 9 months ago by TamAIRC
  0

• difficult to install

  #61 opened 10 months ago by lvwanwan2022
  0

• module 'fastBPE' has no attribute 'fastBPE'

  #57 opened 2 years ago by yuhangjiang22
  2

• Whether to support byte-level-bpe？

  #59 opened 2 years ago by Alex-Songs
  0

• Build fails on Windows 10: Missing `<sys/mman.h>`

  #58 opened 2 years ago by sydrawat01
  3

• 【vocab size】Is there another way to limit the vocab size?

  #47 opened 4 years ago by imyzx2017
  1

• los valores es {suma}") #Se imprime la suma

  #56 opened 3 years ago by miguelangelpro4
  0

• fatal error C1083: Cannot open include file: 'sys/mman.h': No such file or directory

  #40 opened 5 years ago by acmilannesta
  1

• Error occurred when install fastBPE with pip3 on Windows

  #54 opened 3 years ago by hsiang-lee
  1

• core dumped error during learnbpe

  #55 opened 3 years ago by SylasreeKS
  0

• codes_path and vocab_path should be changed to what parameters should be given

  #53 opened 3 years ago by S-GeGe
  1

• Read 311990 words (17104 unique) from text file. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped)

  #52 opened 3 years ago by dinaalaaahmed
  0

• Can fastBPE use the subword codes file?

  #51 opened 3 years ago by SefaZeng
  0

• limitVocab

  #50 opened 3 years ago by geert56
  0

• Cant install fastBPE on windows 10

  #49 opened 4 years ago by shadmanhiya
  1

• python setup.py install failure

  #27 opened 5 years ago by sergeman
  5

• Crash without cython installed !

  #46 opened 4 years ago by QuentinN42
  1

• ReadCodes error

  #35 opened 5 years ago by zhishui3
  2

• Can't install in Ubuntu 20.04

  #45 opened 4 years ago by 1mike12
  1

• fastBPE installation error

  #41 opened 4 years ago by anjalibhavan
  1

• Cant install fastBPE on windows 10

  #44 opened 4 years ago by nag0811
  0

• Still can't install on Win 10

  #42 opened 4 years ago by zslotyi
  0

• unable to compile in windows ,workaround suggested under pytorch / fairseq error #1224 is also not working

  #43 opened 4 years ago by AjitAntony
  0

• [QUESTION] Python Wrapper

  #12 opened 6 years ago by loretoparisi
  6

• Any plans to add BPE-Dropout?

  #39 opened 5 years ago by CyndxAI
  2

• Error while compiling

  #33 opened 5 years ago by aastha19
  1

• Generate release?

  #38 opened 5 years ago by oblute
  0

• Hi, awesome work! But does it support to add special tokens for tokenization?

  #37 opened 5 years ago by jianliu-ml
  0

• Differences with subword-nmt

  #13 opened 5 years ago by loretoparisi
  5

• Question about learnbpe

  #36 opened 5 years ago by dreamingo
  1

• fastBPE install on win10

  #34 opened 5 years ago by GeorgeS2019
  1

• Can I use fastBPE in Chinese?

  #14 opened 6 years ago by JxuHenry
  2

• Error when install with cython

  #28 opened 5 years ago by liug0324
  1

• getvocab issue using french text

  #30 opened 5 years ago by ctoffo
  0

• Python api for learnbpe

  #31 opened 5 years ago by tonylekhtman
  0

• about adding the word blacklist for BPE

  #29 opened 5 years ago by ghchen18
  0

• setup fails when Cython is not available

  #26 opened 5 years ago by ink1
  2

• Nothing happens after applybpe

  #24 opened 5 years ago by kayoyin
  5

• learnbpe and the process is killed

  #22 opened 5 years ago by tuanshanyou
  1

• question about applybpe

  #21 opened 5 years ago by teslacool
  4

• 'ios' file not found

  #18 opened 6 years ago by willy20040711
  2

• Error when using Python API

  #19 opened 6 years ago by sashank06
  1

• fast.cc: No such file or directory

  #20 opened 6 years ago by miaodl
  1

• Verbose mode

  #17 opened 6 years ago by villmow
  1

• Compilation error

  #11 opened 6 years ago by thomwolf
  1

• Issue Importing/Installing Python API on mac OS

  #15 opened 6 years ago by TroyZuroske
  1

• feature request: separate initial loading and per-line operations

  #10 opened 6 years ago by bittlingmayer
  1