kexinhuang12345/MolTrans

About FCS mining

Closed this issue · 1 comments

Hello, thanks for your nice research and package management.
By the way, is there are differences between Byte Pair Encoding and FCS mining algorithm written in paper?

Hi, they are basically the same thing. we use the package of BPE to calculate it but we want to make it more explicitly called so that it is related to the case used here since byte pair encoding is derived from compression.