ovalhub/pyicu

ImportError: dlopen(/usr/local/lib/python2.7/site-packages/_icu.so, 2): Symbol not found: __ZN6icu_638ByteSink15GetAppendBufferEiiPciPi

loretoparisi opened this issue · 13 comments

loretoparisi commented

On macOS, python 2.7.16 I have installed icu4

export ICU_VERSION=64.2
export PYICU_INCLUDES=/usr/local/Cellar/icu4c/64.2/include
export PYICU_LFLAGS=-L/usr/local/Cellar/icu4c/64.2/lib
export PYICU_CFLAGS=-std=c++11
pip install pyicu

but when importing icu I get

>>> import pyicu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pyicu
>>> import pycu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pycu
>>> from icu import UnicodeString, Locale
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "icu/__init__.py", line 37, in <module>
    from _icu import *
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/_icu.so, 2): Symbol not found: __ZN6icu_638ByteSink15GetAppendBufferEiiPciPi
  Referenced from: /usr/local/lib/python2.7/site-packages/_icu.so
  Expected in: flat namespace
 in /usr/local/lib/python2.7/site-packages/_icu.so

I have also installed PyCu version 2.3.1 from sources editing the configuration for macOS:

INCLUDES = {
    'darwin': ['/usr/local/Cellar/icu4c/64.2'],
    'linux': [],
    'freebsd': ['/usr/local/include'],
    'win32': ['c:/icu/include'],
    'sunos5': [],
    'cygwin': [],
}

CFLAGS = {
    'darwin': ['-DPYICU_VER="%s"' %(VERSION), '-std=c++11'],
    'linux': [],
    'freebsd': ['-std=c++11'],
    'win32': ['/Zc:wchar_t', '/EHsc'],
    'sunos5': ['-std=c++11'],
    'cygwin': ['-D_GNU_SOURCE=1', '-std=c++11'],
}

LFLAGS = {
    'darwin': ['-L/usr/local/Cellar/icu4c/64.2/lib'],
    'linux': [],
    'freebsd': ['-L/usr/local/lib'],
    'win32': ['/LIBPATH:c:/icu/lib'],
    'sunos5': [],
    'cygwin': [],
}

LIBRARIES = {
    'darwin': ['/usr/local/Cellar/icu4c/64.2/lib'],
    'linux': [],
    'freebsd': ['icui18n', 'icuuc', 'icudata'],
    'win32': ['icuin', 'icuuc', 'icudt'],
    'sunos5': ['icui18n', 'icuuc', 'icudata'],
    'cygwin': ['icui18n', 'icuuc', 'icudata'],
}

ln -s /usr/local/Cellar/icu4c/64.2/bin/icu-config /usr/local/bin/icu-config
 CFLAGS=-I/usr/local/opt/icu4c/include LDFLAGS=-L/usr/local/opt/icu4c/lib python setup.py build
python setup.py install

resulting in

Installed /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg
Processing dependencies for PyICU==2.3.1
Finished processing dependencies for PyICU==2.3.1

but I'm still getting the same error when importing the class.

Source: https://stackoverflow.com/questions/50217214/import-error-for-icu-in-mac-and-ubuntu-although-pyicu-is-installed-correctly/55905388#55905388

ovalhub commented
  • 👍1
loretoparisi commented

@ovalhub I did some changes, got a different error...

/usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.py:3: UserWarning: Module _icu was already imported from /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.pyc, but /Users/loretoparisi/Documents/Projects/AI/pyicu is being added to sys.path

full stacktrace:

ip-192-168-22-127:pyicu loretoparisi$ python setup.py install

Building PyICU 2.3.1 for ICU 64.2

running install
running bdist_egg
running egg_info
writing PyICU.egg-info/PKG-INFO
writing top-level names to PyICU.egg-info/top_level.txt
writing dependency_links to PyICU.egg-info/dependency_links.txt
reading manifest file 'PyICU.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'PyICU.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.14-x86_64/egg
running install_lib
running build_py
running build_ext
creating build/bdist.macosx-10.14-x86_64/egg
copying build/lib.macosx-10.14-x86_64-2.7/_icu.so -> build/bdist.macosx-10.14-x86_64/egg
copying build/lib.macosx-10.14-x86_64-2.7/PyICU.py -> build/bdist.macosx-10.14-x86_64/egg
creating build/bdist.macosx-10.14-x86_64/egg/icu
copying build/lib.macosx-10.14-x86_64-2.7/icu/__init__.py -> build/bdist.macosx-10.14-x86_64/egg/icu
byte-compiling build/bdist.macosx-10.14-x86_64/egg/PyICU.py to PyICU.pyc
byte-compiling build/bdist.macosx-10.14-x86_64/egg/icu/__init__.py to __init__.pyc
creating stub loader for _icu.so
byte-compiling build/bdist.macosx-10.14-x86_64/egg/_icu.py to _icu.pyc
creating build/bdist.macosx-10.14-x86_64/egg/EGG-INFO
copying PyICU.egg-info/PKG-INFO -> build/bdist.macosx-10.14-x86_64/egg/EGG-INFO
copying PyICU.egg-info/SOURCES.txt -> build/bdist.macosx-10.14-x86_64/egg/EGG-INFO
copying PyICU.egg-info/dependency_links.txt -> build/bdist.macosx-10.14-x86_64/egg/EGG-INFO
copying PyICU.egg-info/top_level.txt -> build/bdist.macosx-10.14-x86_64/egg/EGG-INFO
writing build/bdist.macosx-10.14-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating 'dist/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg' and adding 'build/bdist.macosx-10.14-x86_64/egg' to it
removing 'build/bdist.macosx-10.14-x86_64/egg' (and everything under it)
Processing PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg
Copying PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg to /usr/local/lib/python2.7/site-packages
Adding PyICU 2.3.1 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg
Processing dependencies for PyICU==2.3.1
Finished processing dependencies for PyICU==2.3.1
ip-192-168-22-127:pyicu loretoparisi$ python -c "import icu"
/usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.py:3: UserWarning: Module _icu was already imported from /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.pyc, but /Users/loretoparisi/Documents/Projects/AI/pyicu is being added to sys.path
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "icu/__init__.py", line 37, in <module>
    from _icu import *
  File "build/bdist.macosx-10.14-x86_64/egg/_icu.py", line 7, in <module>
  File "build/bdist.macosx-10.14-x86_64/egg/_icu.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so, 2): Symbol not found: __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
  Referenced from: /Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so
  Expected in: flat namespace
 in /Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so
ovalhub commented
ovalhub commented
loretoparisi commented

@ovalhub so I have checked it out and it seems I have installed this one from brew:

 /usr/local/Cellar/icu4c/64.2

So, asssumed that is correct, I have reverted to the master:

Saved working directory and index state WIP on master: 983144b   - fixed build error with ICU 61, ICU 62   - release PyICU 2.3.1

and tried again

ip-192-168-22-127:pyicu loretoparisi$ python setup.py build
(running 'icu-config --version')

Building PyICU 2.3.1 for ICU 64.2

(running 'icu-config --cxxflags --cppflags')
Adding CFLAGS="-std=c++11 -I/usr/local/Cellar/icu4c/64.2/include" from /usr/local/bin/icu-config
(running 'icu-config --ldflags')
Adding LFLAGS="-L/usr/local/Cellar/icu4c/64.2/lib -licui18n -licuuc -licudata" from /usr/local/bin/icu-config
running build
running build_py
running build_ext

and

python setup.py install
...
PyICU 2.3.1 is already the active version in easy-install.pth

Installed /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg
Processing dependencies for PyICU==2.3.1
Finished processing dependencies for PyICU==2.3.1

but I get again an error:

ip-192-168-22-127:pyicu loretoparisi$ python -c "import icu"
/usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.py:3: UserWarning: Module _icu was already imported from /usr/local/lib/python2.7/site-packages/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg/_icu.pyc, but /Users/loretoparisi/Documents/Projects/AI/pyicu is being added to sys.path
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "icu/__init__.py", line 37, in <module>
    from _icu import *
  File "build/bdist.macosx-10.14-x86_64/egg/_icu.py", line 7, in <module>
  File "build/bdist.macosx-10.14-x86_64/egg/_icu.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so, 2): Symbol not found: __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
  Referenced from: /Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so
  Expected in: flat namespace
 in /Users/loretoparisi/Library/Caches/Python-Eggs/PyICU-2.3.1-py2.7-macosx-10.14-x86_64.egg-tmp/_icu.so
ovalhub commented
  • 👍1
loretoparisi commented

[UPDATE]
Sorry for the long thread, here it is a cleaned up example:

ip-192-168-22-127:AI loretoparisi$ brew uninstall --ignore-dependencies icu4c
Uninstalling /usr/local/Cellar/icu4c/64.2... (257 files, 69.2MB)
ip-192-168-22-127:AI loretoparisi$ brew install icu4c
Updating Homebrew...
==> Downloading https://homebrew.bintray.com/bottles/icu4c-64.2.mojave.bottle.tar.gz
Already downloaded: /Users/loretoparisi/Library/Caches/Homebrew/downloads/85517172585f53ec6345bbab70673ae252f1fb92de862d669bd47db314e40162--icu4c-64.2.mojave.bottle.tar.gz
==> Pouring icu4c-64.2.mojave.bottle.tar.gz
==> Caveats
icu4c is keg-only, which means it was not symlinked into /usr/local,
because macOS provides libicucore.dylib (but nothing else).

If you need to have icu4c first in your PATH run:
  echo 'export PATH="/usr/local/opt/icu4c/bin:$PATH"' >> ~/.bash_profile
  echo 'export PATH="/usr/local/opt/icu4c/sbin:$PATH"' >> ~/.bash_profile

For compilers to find icu4c you may need to set:
  export LDFLAGS="-L/usr/local/opt/icu4c/lib"
  export CPPFLAGS="-I/usr/local/opt/icu4c/include"

For pkg-config to find icu4c you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/icu4c/64.2: 257 files, 69.2MB
ip-192-168-22-127:AI loretoparisi$ ls /usr/local/Cellar/icu4c/
64.2
ip-192-168-22-127:AI loretoparisi$ export ICU_VERSION=64
ip-192-168-22-127:AI loretoparisi$ export PYICU_INCLUDES=/usr/local/Cellar/icu4c/64.2/include
ip-192-168-22-127:AI loretoparisi$ export PYICU_LFLAGS=-L/usr/local/Cellar/icu4c/64.2/lib
ip-192-168-22-127:AI loretoparisi$ python -c "import icu"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named icu
ip-192-168-22-127:AI loretoparisi$ pip install pyicu
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pyicu
Installing collected packages: pyicu
Successfully installed pyicu-2.3.1
ip-192-168-22-127:AI loretoparisi$ python -c "import icu"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/icu/__init__.py", line 37, in <module>
    from _icu import *
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/_icu.so, 2): Symbol not found: __ZN6icu_638ByteSink15GetAppendBufferEiiPciPi
  Referenced from: /usr/local/lib/python2.7/site-packages/_icu.so
  Expected in: flat namespace
 in /usr/local/lib/python2.7/site-packages/_icu.so
ovalhub commented
loretoparisi commented

@ovalhub thank you Andi, I will try to get rid then to clean up all attempts. Thank you very much.

loretoparisi commented

So thank you very much, could you be so kind to confirm me these steps:

  1. I have removed the current PyICU install first - pip uninstall pyicu
Skipping icu as it is not installed.
ip-192-168-22-127:AI loretoparisi$ pip uninstall pyicu
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Uninstalling PyICU-2.3.1:
  Would remove:
    /usr/local/lib/python2.7/site-packages/PyICU-2.3.1.dist-info/*
    /usr/local/lib/python2.7/site-packages/PyICU.py
    /usr/local/lib/python2.7/site-packages/_icu.so
    /usr/local/lib/python2.7/site-packages/icu/*
Proceed (y/n)? y
  Successfully uninstalled PyICU-2.3.1

  1. I have removed any previous installation of icu4c, since I'm using Homebrew the command was brew uninstall --ignore-dependencies icu4c, so therefore I have a No such keg: /usr/local/Cellar/icu4c on my macOS at this point.

  2. Cleaning the Homebrew cache, to avoid any possibile previous install - brew cleanup

  3. install icu4c again - brew install icu4c & brew link icu4c --force

ip-192-168-22-157:~ loretoparisi$ brew install icu4c
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
node ✔                    diff-pdf                  pdftoipe                  pmd                       sipsak                    sourcekitten              vulkan-headers
poppler ✔                 joplin                    plantuml                  shpotify                  sonobuoy                  txr                       youtube-dl

==> Downloading https://homebrew.bintray.com/bottles/icu4c-64.2.mojave.bottle.tar.gz
Already downloaded: /Users/loretoparisi/Library/Caches/Homebrew/downloads/85517172585f53ec6345bbab70673ae252f1fb92de862d669bd47db314e40162--icu4c-64.2.mojave.bottle.tar.gz
==> Pouring icu4c-64.2.mojave.bottle.tar.gz
==> Caveats
icu4c is keg-only, which means it was not symlinked into /usr/local,
because macOS provides libicucore.dylib (but nothing else).

If you need to have icu4c first in your PATH run:
  echo 'export PATH="/usr/local/opt/icu4c/bin:$PATH"' >> ~/.bash_profile
  echo 'export PATH="/usr/local/opt/icu4c/sbin:$PATH"' >> ~/.bash_profile

For compilers to find icu4c you may need to set:
  export LDFLAGS="-L/usr/local/opt/icu4c/lib"
  export CPPFLAGS="-I/usr/local/opt/icu4c/include"

For pkg-config to find icu4c you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/icu4c/64.2: 257 files, 69.2MB
  1. install and skip cache - CFLAGS=-I/usr/local/opt/icu4c/include PYICU_LFLAGS=-L/usr/local/Cellar/icu4c/64.2/lib PYICU_CFLAGS="-std=c++11" ICU_VERSION=64.2 pip --no-cache-dir install PyICU
ip-192-168-22-157:~ loretoparisi$ CFLAGS=-I/usr/local/opt/icu4c/include PYICU_LFLAGS=-L/usr/local/Cellar/icu4c/64.2/lib PYICU_CFLAGS="-std=c++11" ICU_VERSION=64.2 pip --no-cache-dir install PyICU
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting PyICU
  Downloading https://files.pythonhosted.org/packages/e9/35/211ffb949c68e688ade7d40426de030a24eaec4b6c45330eeb9c0285f43a/PyICU-2.3.1.tar.gz (214kB)
    100% |████████████████████████████████| 215kB 921kB/s 
Installing collected packages: PyICU
  Running setup.py install for PyICU ... done
Successfully installed PyICU-2.3.1
  1. check the static library is there in **/usr/local/lib/python2.7/site-packages/_icu.so **
ip-192-168-22-157:~ loretoparisi$ ls -l /usr/local/lib/python2.7/site-packages/_icu.so 
-rwxr-xr-x  1 loretoparisi  admin  1282908 30 Apr 15:09 /usr/local/lib/python2.7/site-packages/_icu.so
  1. check if the symbol __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi is there! nm /usr/local/lib/python2.7/site-packages/_icu.so | grep __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
ip-192-168-22-157:~ loretoparisi$ nm /usr/local/lib/python2.7/site-packages/_icu.so | grep __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
                 U __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
  1. I run and export the library path DYLD_LIBRARY_PATH=/usr/local/Cellar/icu4c/64.2/lib/ python -c "import icu", At this point I do not know where I am wrong, since I get the same error Symbol not found: __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi even if that symbol is there!

Thank you very much for your help.

loretoparisi commented

[UPDATE]

Ok, it worked ONLY in this way!

curl -LO http://download.icu-project.org/files/icu4c/55.1/icu4c-55_1-src.tgz
tar xzvf icu4c-55_1-src.tgz
cd icu/source
chmod +x runConfigureICU configure install-sh
./runConfigureICU MacOSX
make
sudo make install
sudo cp -r common/unicode /usr/local/include/

and then

ICU_VERSION=55.1 pip --no-cache-dir install PyICU
python -c "from icu import Locale; locale = Locale('ta_TAM'); name = locale.getDisplayName(); print(name)"
tamil (TAM)

It's crazy, I do not know why, but it worked from the sources with the version 55.1. I think there is an issue somewhere in any case, because there are a lot of complains.
Thank you.

ovalhub commented
loretoparisi commented

Thank you very much for your help!

  • 👍1