pyfasttext predict sometimes return empty result
Opened this issue · 2 comments
When I use the pyfasttext with a same model, to predict this:
dwell dwelluk -> [u'company']
Joshua Katz joshuakatz4 -> [u'male']
meredith mears mered148r -> [u'female']
Shenia Pellum SheniaPellum -> [u'female']
Matt Inama mattinama -> [u'male']
maureen russell mopruss -> [u'male']
RickLeeJames RickLeeJames -> []
Lauren Townsend laurenttownsend -> [u'female']
Andy Campbell lordsnarfdog -> [u'male']
Mark Newham mhn56 -> [u'male']
Ryan DatBirdbrain -> [u'male']
MaryAnn maryannstewart1 -> [u'female']
Career & Business HustleNaija -> [u'company']
Kevin A. Thompson kevinathompson -> [u'male']
Kevin Aston AstonFarms -> [u'male']
while if I use the command line, the result will be:
__label__female
__label__male
__label__female
__label__female
__label__male
__label__female
__label__female --> missing in the above
__label__female
__label__male
__label__male
__label__male
__label__female
__label__company
__label__male
__label__male
So the results from pyfasttext and command line are different and missing (in case of pyfasttext)
Do you have any idea why it's different?
Hi @nguyenhoan1988
pyfasttext
uses the same source code as the fastText
binary (with a thin Python wrapper).
So, there should be no difference... But fastText
adds a newline at the end of each sentence, so you may have to add '\n' manually to your lines.
If that does not work, can you upload your model?
@vrasneur thank you so much
I fixed the problem by adding '\n' to every single line before calling predict
Maybe you should add this warning to the README
or automatically add '\n' when predict
is called