About FCS mining
Closed this issue · 1 comments
hyojin0912 commented
Hello, thanks for your nice research and package management.
By the way, is there are differences between Byte Pair Encoding and FCS mining algorithm written in paper?
kexinhuang12345 commented
Hi, they are basically the same thing. we use the package of BPE to calculate it but we want to make it more explicitly called so that it is related to the case used here since byte pair encoding is derived from compression.