ntedgi/node-efficientnet

Compress and decompress misc json file with snappy compression

ntedgi opened this issue · 9 comments

ntedgi commented

After adding a lot of translations files the package size increase to ~600KB .

We need to compress to json file and to decompress them at runtime i prefer using snappy compression

then when loading a specific language provider we only decompress is language misc file

rokaicker commented

Hey @ntedgi, if nobody is working on this, is it okay if I give it a shot? (I am not particalurly well-versed in machine learning, but this seems like something I might be able to help with)

But just to confirm - this is to compress the language files found here.

Then decompress the specific language that is specified by the "locale" key in the result object?

  • 👍1
ntedgi commented

Go for it thanks

PranavBhatS commented

@ntedgi anyone currently working on this, if not can i pick up this issue?

ntedgi commented

Go for it

  • ❤️1
SananditaDas commented

Hey @ntedgi Can I pick up this issue if its not resolved yet?
Also I wanted to know, are you aiming for static and compressed translation files before uncompressing them at runtime?

  • 👍1
SananditaDas commented

Hey @ntedgi Can I pick up this issue if its not resolved yet? Also I wanted to know, are you aiming for static and compressed translation files before uncompressing them at runtime?

I have replaced the json translation files with compressed binary-s (using snappy), and the code is working fine while decompressing the lang file at runtime.
But some of the testcases are failing, which are invoking the remote model.
Could you help?

ntedgi commented
  • Update remote files links to the relative path at your working branch
  • fix the falling test
  • after test pass and mearge to master
  • Change test files to relative path at 'main' branch
SananditaDas commented
  • Update remote files links to the relative path at your working branch
  • fix the falling test
  • after test pass and mearge to master
  • Change test files to relative path at 'main' branch

Means, I need to change the remote model url to localModelRootDirectory , right? Then after merging with main branch, change it back to the original url?

ntedgi commented