ARBML/tnkeeh

Dictionaries folder doesn't get downloaded when downloading the package

Closed this issue · 4 comments

malhomaid commented

Hello,

The dictionaries folder doesn't get downloaded when downloading the package had to download it manually inside site-packages/tnkeeh/ to get normalize option to work.

---------------------------------------------------------------------------
~/.local/share/virtualenvs/insight-Pf6tRULh/lib/python3.8/site-packages/tnkeeh/tnkeeh.py in _normalize_data(text)
     18     rel_path = os.path.dirname(__file__)
     19     norm_dict_path = os.path.join(rel_path, "dictionaries/norm_dict.pl")
---> 20     norm_dict = pickle.load(open(norm_dict_path, "rb"))
     21     # use a mapping dictionary
     22     regex = re.compile("|".join(map(re.escape, norm_dict.keys())))

FileNotFoundError: [Errno 2] No such file or directory: '/home/mohammad/.local/share/virtualenvs/insight-Pf6tRULh/lib/python3.8/site-packages/tnkeeh/dictionaries/norm_dict.pl'
zaidalyafeai commented

thanks @mhomaid1 for reporting that. @MagedSaeed any suggestions ?

MagedSaeed commented

Hey @zaidalyafeai , @mhomaid1

Well, what we can do is we can upload these dictionaries to a serving domain like CDN or GitHub releases. GitHub release seems to be more convenient. We can, then, implement a download() function that talks to this domain and download the files to its path inside site-packages.

Furhtermore, we can achieve faster download time by using this library https://github.com/halilozercan/pget. I, personally, did not use it before but wanted to.

Please let me know if this sounds ok.

zaidalyafeai commented

Sounds good

zaidalyafeai commented

@mhomaid1 it should work now in the latest pip version.