stepthom/lucene-lda

How to transform the data from mallet to one that can be used by this tool?

rrgirish opened this issue · 2 comments

rrgirish commented

We are able to get Lucene-LDA to compile by removing the lucene-3.0 Jar(and leave the 3.5 jar) from the lib directory.

However, when we try to run the indexDirectory command on the documents that we have, we observed that as per the readme and the source code, lucene-lda doesn't run MALLET by itself.

So we ran mallet on the data first and obtained the output from MALLET. However, after this Lucene-lda doesn't recognize the output from the mallet file(when we try to run the queryWithLDA. command). Does this need to be in some specific data format?

stepthom commented

lucene-lda doesn't read in MALLET files directly, at the moment. MALLET output files need to be preprocessed (e.g., using a simple script) to create the four input files described in the README, under the bullet "You have already executed LDA..."

rrgirish commented

Yup.. figured that out. Thanks!