“'gcc' failed with exit status 1” when trying to install Mecab with PyPy docker image

Question

“'gcc' failed with exit status 1” when trying to install Mecab with PyPy docker image

starmandeluxe opened this issue 4 years ago · 1 comments

My Docker image builds just fine when trying to install Mecab with CPython using image tag python:3.8-slim, but it fails with PyPy.

My Dockerfile:

FROM pypy:3-7

RUN pypy -m ensurepip --default-pip

ENV PYTHONDONTWRITEBYTECODE 1
ENV FLASK_APP "main.py"
ENV PYTHONUNBUFFERED 1

RUN mkdir /app
WORKDIR /app

# Install Mecab
RUN apt-get update && apt-get -y install mecab libmecab-dev mecab-ipadic-utf8 git make curl xz-utils file sudo

# Set up Mecab
RUN git clone --depth 1 https://github.com/neologd/mecab-ipadic-neologd.git
RUN echo yes | mecab-ipadic-neologd/bin/install-mecab-ipadic-neologd -n -a

COPY Pip* /app/

RUN pip install --upgrade pip && \
    pip install pipenv && \
    pipenv install --dev --system --deploy --ignore-pipfile

ADD . /app

And the full error output:

Installing initially failed dependencies...
[InstallError]:   File "/opt/pypy/site-packages/pipenv/cli/command.py", line 253, in install
[InstallError]:       site_packages=state.site_packages
[InstallError]:   File "/opt/pypy/site-packages/pipenv/core.py", line 2063, in do_install
[InstallError]:       keep_outdated=keep_outdated
[InstallError]:   File "/opt/pypy/site-packages/pipenv/core.py", line 1312, in do_init
[InstallError]:       pypi_mirror=pypi_mirror,
[InstallError]:   File "/opt/pypy/site-packages/pipenv/core.py", line 900, in do_install_dependencies
[InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]:   File "/opt/pypy/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/opt/pypy/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting mecab-python3==0.996.5
[pipenv.exceptions.InstallError]:   Using cached mecab-python3-0.996.5.tar.gz (65 kB)
[pipenv.exceptions.InstallError]: Building wheels for collected packages: mecab-python3
[pipenv.exceptions.InstallError]:   Building wheel for mecab-python3 (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for mecab-python3 (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   ERROR: Command errored out with exit status 1:
[pipenv.exceptions.InstallError]:    command: /opt/pypy/bin/pypy3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-9i8vw475
[pipenv.exceptions.InstallError]:        cwd: /tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/
[pipenv.exceptions.InstallError]:   Complete output (59 lines):
[pipenv.exceptions.InstallError]:   running bdist_wheel
[pipenv.exceptions.InstallError]:   running build
[pipenv.exceptions.InstallError]:   running build_py
[pipenv.exceptions.InstallError]:   creating build
[pipenv.exceptions.InstallError]:   creating build/lib.linux-x86_64-3.6
[pipenv.exceptions.InstallError]:   creating build/lib.linux-x86_64-3.6/MeCab
[pipenv.exceptions.InstallError]:   copying src/MeCab/__init__.py -> build/lib.linux-x86_64-3.6/MeCab
[pipenv.exceptions.InstallError]:   warning: build_py: byte-compiling is disabled, skipping.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   running build_ext
[pipenv.exceptions.InstallError]:   Extension build configuration adjusted:
[pipenv.exceptions.InstallError]:    include_dirs = ['/usr/include']
[pipenv.exceptions.InstallError]:    library_dirs = ['/usr/lib/x86_64-linux-gnu']
[pipenv.exceptions.InstallError]:    libraries    = ['mecab', 'stdc++']
[pipenv.exceptions.InstallError]:    swig_opts    = ['-O', '-builtin', '-c++', '-py3', '-I/usr/include']
[pipenv.exceptions.InstallError]:   building 'MeCab._MeCab' extension
[pipenv.exceptions.InstallError]:   swigging src/MeCab/MeCab.i to src/MeCab/MeCab_wrap.cpp
[pipenv.exceptions.InstallError]:   swig -python -O -builtin -c++ -py3 -I/usr/include -o src/MeCab/MeCab_wrap.cpp src/MeCab/MeCab.i
[pipenv.exceptions.InstallError]:   /usr/include/mecab.h:136: Warning 302: Identifier 'surface' redefined by %extend (ignored),
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab.i:74: Warning 302: %extend definition of 'surface'.
[pipenv.exceptions.InstallError]:   /usr/include/mecab.h:848: Warning 302: Identifier 'set_sentence' redefined by %extend (ignored),
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab.i:105: Warning 302: %extend definition of 'set_sentence'.
[pipenv.exceptions.InstallError]:   creating build/temp.linux-x86_64-3.6
[pipenv.exceptions.InstallError]:   creating build/temp.linux-x86_64-3.6/src
[pipenv.exceptions.InstallError]:   creating build/temp.linux-x86_64-3.6/src/MeCab
[pipenv.exceptions.InstallError]:   gcc -pthread -DNDEBUG -O2 -fPIC -I/usr/include -I/opt/pypy/include -c src/MeCab/MeCab_wrap.cpp -o build/temp.linux-x86_64-3.6/src/MeCab/MeCab_wrap.o -Wno-unused-variable
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp: In function ‘void SwigPyBuiltin_SetMetaType(PyTypeObject*, PyTypeObject*)’:
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:3444:11: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘ob_base’; did you mean ‘tp_base’?
[pipenv.exceptions.InstallError]:        type->ob_base.ob_base.ob_type = metatype;
[pipenv.exceptions.InstallError]:              ^~~~~~~
[pipenv.exceptions.InstallError]:              tp_base
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp: At global scope:
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:8464:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:8697:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:8978:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:9223:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:9445:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:9681:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:    };
[pipenv.exceptions.InstallError]:    ^
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp: In function ‘PyObject* PyInit__MeCab()’:
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:10401:16: error: ‘PyDescr_NewGetSet’ was not declared in this scope
[pipenv.exceptions.InstallError]:      this_descr = PyDescr_NewGetSet(SwigPyObject_type(), &this_getset_def);
[pipenv.exceptions.InstallError]:                   ^~~~~~~~~~~~~~~~~
[pipenv.exceptions.InstallError]:   src/MeCab/MeCab_wrap.cpp:10401:16: note: suggested alternative: ‘PyDescrObject’
[pipenv.exceptions.InstallError]:      this_descr = PyDescr_NewGetSet(SwigPyObject_type(), &this_getset_def);
[pipenv.exceptions.InstallError]:                   ^~~~~~~~~~~~~~~~~
[pipenv.exceptions.InstallError]:                   PyDescrObject
[pipenv.exceptions.InstallError]:   error: command 'gcc' failed with exit status 1
[pipenv.exceptions.InstallError]:   ----------------------------------------
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for mecab-python3
[pipenv.exceptions.InstallError]:   Running setup.py clean for mecab-python3
[pipenv.exceptions.InstallError]: Failed to build mecab-python3
[pipenv.exceptions.InstallError]: Installing collected packages: mecab-python3
[pipenv.exceptions.InstallError]:     Running setup.py install for mecab-python3: started
[pipenv.exceptions.InstallError]:     Running setup.py install for mecab-python3: finished with status 'error'
[pipenv.exceptions.InstallError]:     ERROR: Command errored out with exit status 1:
[pipenv.exceptions.InstallError]:      command: /opt/pypy/bin/pypy3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-784d25jp/install-record.txt --single-version-externally-managed --compile --install-headers /opt/pypy/include/mecab-python3
[pipenv.exceptions.InstallError]:          cwd: /tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/
[pipenv.exceptions.InstallError]:     Complete output (60 lines):
[pipenv.exceptions.InstallError]:     running install
[pipenv.exceptions.InstallError]:     running build
[pipenv.exceptions.InstallError]:     running build_py
[pipenv.exceptions.InstallError]:     creating build
[pipenv.exceptions.InstallError]:     creating build/lib.linux-x86_64-3.6
[pipenv.exceptions.InstallError]:     creating build/lib.linux-x86_64-3.6/MeCab
[pipenv.exceptions.InstallError]:     copying src/MeCab/__init__.py -> build/lib.linux-x86_64-3.6/MeCab
[pipenv.exceptions.InstallError]:     copying src/MeCab/MeCab.py -> build/lib.linux-x86_64-3.6/MeCab
[pipenv.exceptions.InstallError]:     warning: build_py: byte-compiling is disabled, skipping.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     running build_ext
[pipenv.exceptions.InstallError]:     Extension build configuration adjusted:
[pipenv.exceptions.InstallError]:      include_dirs = ['/usr/include']
[pipenv.exceptions.InstallError]:      library_dirs = ['/usr/lib/x86_64-linux-gnu']
[pipenv.exceptions.InstallError]:      libraries    = ['mecab', 'stdc++']
[pipenv.exceptions.InstallError]:      swig_opts    = ['-O', '-builtin', '-c++', '-py3', '-I/usr/include']
[pipenv.exceptions.InstallError]:     building 'MeCab._MeCab' extension
[pipenv.exceptions.InstallError]:     swigging src/MeCab/MeCab.i to src/MeCab/MeCab_wrap.cpp
[pipenv.exceptions.InstallError]:     swig -python -O -builtin -c++ -py3 -I/usr/include -o src/MeCab/MeCab_wrap.cpp src/MeCab/MeCab.i
[pipenv.exceptions.InstallError]:     /usr/include/mecab.h:136: Warning 302: Identifier 'surface' redefined by %extend (ignored),
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab.i:74: Warning 302: %extend definition of 'surface'.
[pipenv.exceptions.InstallError]:     /usr/include/mecab.h:848: Warning 302: Identifier 'set_sentence' redefined by %extend (ignored),
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab.i:105: Warning 302: %extend definition of 'set_sentence'.
[pipenv.exceptions.InstallError]:     creating build/temp.linux-x86_64-3.6
[pipenv.exceptions.InstallError]:     creating build/temp.linux-x86_64-3.6/src
[pipenv.exceptions.InstallError]:     creating build/temp.linux-x86_64-3.6/src/MeCab
[pipenv.exceptions.InstallError]:     gcc -pthread -DNDEBUG -O2 -fPIC -I/usr/include -I/opt/pypy/include -c src/MeCab/MeCab_wrap.cpp -o build/temp.linux-x86_64-3.6/src/MeCab/MeCab_wrap.o -Wno-unused-variable
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp: In function ‘void SwigPyBuiltin_SetMetaType(PyTypeObject*, PyTypeObject*)’:
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:3444:11: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘ob_base’; did you mean ‘tp_base’?
[pipenv.exceptions.InstallError]:          type->ob_base.ob_base.ob_type = metatype;
[pipenv.exceptions.InstallError]:                ^~~~~~~
[pipenv.exceptions.InstallError]:                tp_base
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp: At global scope:
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:8464:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:8697:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:8978:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:9223:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:9445:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:9681:1: error: too many initializers for ‘PyHeapTypeObject’ {aka ‘_heaptypeobject’}
[pipenv.exceptions.InstallError]:      };
[pipenv.exceptions.InstallError]:      ^
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp: In function ‘PyObject* PyInit__MeCab()’:
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:10401:16: error: ‘PyDescr_NewGetSet’ was not declared in this scope
[pipenv.exceptions.InstallError]:        this_descr = PyDescr_NewGetSet(SwigPyObject_type(), &this_getset_def);
[pipenv.exceptions.InstallError]:                     ^~~~~~~~~~~~~~~~~
[pipenv.exceptions.InstallError]:     src/MeCab/MeCab_wrap.cpp:10401:16: note: suggested alternative: ‘PyDescrObject’
[pipenv.exceptions.InstallError]:        this_descr = PyDescr_NewGetSet(SwigPyObject_type(), &this_getset_def);
[pipenv.exceptions.InstallError]:                     ^~~~~~~~~~~~~~~~~
[pipenv.exceptions.InstallError]:                     PyDescrObject
[pipenv.exceptions.InstallError]:     error: command 'gcc' failed with exit status 1
[pipenv.exceptions.InstallError]:     ----------------------------------------
[pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 1: /opt/pypy/bin/pypy3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"'; __file__='"'"'/tmp/pip-install-7gu0in6y/mecab-python3_30c04743845644dab48b7bc8db7a8877/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-784d25jp/install-record.txt --single-version-externally-managed --compile --install-headers /opt/pypy/include/mecab-python3 Check the logs for full command output.
ERROR: Couldn't install package: mecab-python3
 Package installation failed...

I tried adding gcc to the installation but it did not solve the problem. Is Mecab not compatible with PyPy?

Answer 1 · 2021-02-08T11:13:23.000Z

You are having issues with mecab-python3, which is not maintained here but at the repository below.

https://github.com/SamuraiT/mecab-python3