coady/lupyne

indexer.commit get struck when using multiprocess

khasa3 opened this issue · 1 comments

when indexer.commit() is run using a process (multiprocess), commit tends to get struck.
I've tried attachCurrentThread() as well, but it doesnt seem to work.

Is there any way where i ll be able to use multiprocess along with lypyne

Following is the code:

import lucene
from lupyne import engine
lucene.initVM()
#assert lucene.getVMEnv() or lucene.initVM()
from multiprocessing import Process

#vm_env = lucene.initVM(vmargs=['-Djava.awt.headless=true'])
#from org.apache.lucene import analysis, document, index, queryparser, search, store, util
class testd:
def idx(self):
#lucene.getVMEnv().attachCurrentThread()
print("init")
indexer = engine.Indexer()
indexer.set('fieldname', stored=True) # settings for all documents of indexer; indexed and tokenized is the default
indexer.add(fieldname="sample_test")
print("Trying to commit")
indexer.commit()
print("done")

if __name__ == '__main__':
#testd().idx()
p = Process(target=testd().idx)
p.start()
p.join()

coady commented

lucene.initVM() has to be called exactly once in the process. Try moving lucene.initVM() from the module to inside the idx method.

Note lucene only supports one writer per index at a time, if that's your ultimate goal.