Compress tokenized strings
Opened this issue · 0 comments
eried commented
Hi,
For compressing tokenized strings similar to:
//HEADER//CODE1//decimals_here//CODE4//decimals_here//CODE3//decimals_here//CODE5//decimals_here//ENDING//
Do I need to split them on the slashes for it to be effective? I ran the model generator with a giant example with lots of these and the output model does not actually compresses anything, I see the last array fill with 00s
Any hint? thanks!