Model is not cross-platform
Davipb opened this issue · 2 comments
The generated model file isn't cross platform: Trying to load a model file generated in a Windows computer on a Linux computer or vice-versa causes JamSpell to hang forever.
Here are the files you can use to reproduce the issue. The windows.bin model was trained on Windows 10 64-bit, and the linux.bin model was trained on Ubuntu 18.04 64-bit (through the Windows WSL), both using the provided alphabet.txt and data.txt
I've also stumbled at this problem. The issue is with wchar_t which is 2 bytes in Windows and 4 bytes in Linux. HandyPack serializer uses sizeof(T) to determine the size of chunk to read and write, so there is no way to read and write models across platform correctly.
The workaround might be to create a specialized version of PodSerializer specifically for wchar_t with fixed chunk size.
Fixed in "Pro" version. Now model can load / train anywhere (windows / mac / linux / mobile devices).
JamSpellPro is available at jamspell.com