ImportError: cannot import name 'logsumexp'
Closed this issue · 11 comments
I know this code hasn't been maintained in a while, but I'm trying to build a container around it.
I'm having issues getting it to run though;
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/subjectivity_classifier/subjectivity/classify.py", line 39, in <module>
print_sentences(_get_subj_or_obj_sentences_from_text(sentence))
File "/subjectivity_classifier/subjectivity/classify.py", line 20, in _get_subj_or_obj_sentences_from_text
from subjectivity.subjectivity_classifier import SubjectivityClassifier
File "/subjectivity_classifier/subjectivity/subjectivity_classifier.py", line 2, in <module>
from gensim.models import KeyedVectors
File "/usr/local/lib/python3.6/site-packages/gensim/__init__.py", line 6, in <module>
from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization, utils # noqa:F401
File "/usr/local/lib/python3.6/site-packages/gensim/models/__init__.py", line 8, in <module>
from .hdpmodel import HdpModel # noqa:F401
File "/usr/local/lib/python3.6/site-packages/gensim/models/hdpmodel.py", line 46, in <module>
from gensim.models import basemodel, ldamodel
File "/usr/local/lib/python3.6/site-packages/gensim/models/ldamodel.py", line 51, in <module>
from scipy.misc import logsumexp
ImportError: cannot import name 'logsumexp'
p.s: Just already tried to compile it with older versions released around the time this code was made , but doesn't work unfortunately;
tensorflow==1.12.0
numpy==1.15.0
gensim==3.0.0
nltk==3.4.1
jupyter ???
matplotlib==3.0.2
Hi Marijn,
Thank you for reaching out. This was a weekend project I wrote a few years ago and did not expect the popularity it reached. Had I known I would have done a better job packaging it! Now I have almost no time. My apologies.
Regarding your issue, this seems to be connected to gensim==3.0.0 using a "new" version of scipy. I wonder if you can install an older version of scipy with misc.logsumexp.
Have a look at this issue
https://stackoverflow.com/questions/43037903/importerror-cannot-import-name-logsumexp-when-importing-sklearn-model-selecti
What happens if you do
pip uninstall scikit-learn
pip install scikit-learn==0.18.2Thanks for responding! Unfortunately now I'm getting compilation errors (after following the SO article);
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/subjectivity_classifier/subjectivity/classify.py", line 39, in <module>
print_sentences(_get_subj_or_obj_sentences_from_text(sentence))
File "/subjectivity_classifier/subjectivity/classify.py", line 20, in _get_subj_or_obj_sentences_from_text
from subjectivity.subjectivity_classifier import SubjectivityClassifier
File "/subjectivity_classifier/subjectivity/subjectivity_classifier.py", line 2, in <module>
from gensim.models import KeyedVectors
File "/usr/local/lib/python3.6/site-packages/gensim/__init__.py", line 6, in <module>
from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization, utils # noqa:F401
File "/usr/local/lib/python3.6/site-packages/gensim/models/__init__.py", line 7, in <module>
from .coherencemodel import CoherenceModel # noqa:F401
File "/usr/local/lib/python3.6/site-packages/gensim/models/coherencemodel.py", line 29, in <module>
from gensim.topic_coherence import (segmentation, probability_estimation,
File "/usr/local/lib/python3.6/site-packages/gensim/topic_coherence/probability_estimation.py", line 14, in <module>
from gensim.topic_coherence.text_analysis import (
File "/usr/local/lib/python3.6/site-packages/gensim/topic_coherence/text_analysis.py", line 23, in <module>
from gensim.models.word2vec import Word2Vec
File "/usr/local/lib/python3.6/site-packages/gensim/models/word2vec.py", line 131, in <module>
from gensim.models.word2vec_inner import train_batch_sg, train_batch_cbow
File "__init__.pxd", line 861, in init gensim.models.word2vec_inner (./gensim/models/word2vec_inner.c:11403)
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
Because sharing is caring, here's the docker file I'm using;
# Docker file for a CentOS-based Python3 image
FROM python:3.6-buster
# Download the git repo
RUN apt-get install -y git
RUN git clone https://github.com/fractalego/subjectivity_classifier.git
# Replace requirements from repo with versioned packages that where around when the original dev made this code.
RUN cd subjectivity_classifier && echo "" > requirements.txt
RUN cd subjectivity_classifier && echo 'tensorflow==1.12.0\nnumpy==1.15.0\ngensim==3.0.0\nnltk==3.4.1\njupyter\nmatplotlib==3.0.2' > requirements.txt
# Download requirements
RUN cd subjectivity_classifier && pip install -r requirements.txt
# Uninstall scikitlearn according to Github Issue
RUN cd subjectivity_classifier && pip uninstall scikit-learn && pip install scikit-learn==0.18.2
# Download Dataset
RUN cd subjectivity_classifier/data/word_embeddings/ && wget -nc http://nlulite.com/download/glove
WORKDIR /subjectivity_classifier
#Now use with -m subjectivity.classify this is a test.
ENTRYPOINT ["python"]
Can you use numpy==1.16.0? It should work
Now I'm getting the exception I got the first time.
I have updated the requirements and it seems to work now. Docker works with the current requirements.txt
You could do a PR with your Dockerfile, it seems fair you would get the credit 😄
Awesome man, thanks. I'm almost there; nltk needs a "punkt" library so it seems.
Python isnt my "native" language, so I'll leave the details to you;
It's missing
nltk.download('punkt')
(I added that underneath the import statements but I guess you would foresee a better location for it ;) )
:)
This is the Dockerfile I am using
FROM python:3.6-buster
# Download the git repo
RUN apt-get install -y git
RUN git clone https://github.com/fractalego/subjectivity_classifier.git
# Download requirements
RUN cd subjectivity_classifier && pip install -r requirements.txt
# Download Dataset
RUN cd subjectivity_classifier/data/word_embeddings/ && wget -nc http://nlulite.com/download/glove -O glove.6B.50d.txt
RUN python -m nltk.downloader 'punkt'
WORKDIR /subjectivity_classifier
#Now use with -m subjectivity.classify this is a test.
ENTRYPOINT ["python"]This is really an old fashioned way of doing NLP! It would be really cool to update the system with more modern techniques. One day when I have the time...
Submitted PR #3 : It's the dockerfile I made and the nltk punkt library upgrade download which was needed.
But maybe the RUN python -m nltk.downloader 'punkt' would suffice as well. I'll leave it for you to judge.
This is really an old fashioned way of doing NLP! It would be really cool to update the system with more modern techniques. One day when I have the time...
I would be your first "customer" ;-) 👍