dmarcous/spark-glove

numberOfPartitions can be 0 if wordListSize is small

Opened this issue · 0 comments

luyi666 commented 
val wordListSize = wordsInVocabulary.count
val numberOfPartitions = Math.floor(wordListSize/OVERFLOW_LENGTH.toDouble).toInt

In Cooccur.scala,
numberOfPartitions can be 0 if wordListSize is less than OVERFLOW_LENGTH, which leads to an error.