Ed-von-Schleck/shoco

Compress tokenized strings

Opened this issue · 0 comments

eried commented

Hi,

For compressing tokenized strings similar to:

//HEADER//CODE1//decimals_here//CODE4//decimals_here//CODE3//decimals_here//CODE5//decimals_here//ENDING//

Do I need to split them on the slashes for it to be effective? I ran the model generator with a giant example with lots of these and the output model does not actually compresses anything, I see the last array fill with 00s

Any hint? thanks!