Multithreaded spaCy
asitemade4u opened this issue · 1 comments
asitemade4u commented
First, let me thank you for this comprehensive, outstanding package!
Then, as I compared cleanNLP with Ken Benoit's spaCy package, I noticed his package allowed fine tuning of the multithreading of spaCy.
As I have to treat quite a big corpus, I was wondering if that was also possible from cleanNLP.
statsmaths commented
Unfortunately this is no longer possible given updates to space. From here:
The keyword argument n_threads on the .pipe methods is now deprecated,
as the v2.x models cannot release the global interpreter lock. (Future versions
may introduce a n_process argument for parallel inference via multiprocessing.)
As soon as a future version includes this I will be happy to link to it through cleanNLP.