Cannot install python-olm on Windows via pip (Py38 amd64)
Opened this issue · 10 comments
C:\>pip install python-olm
Collecting python-olm
Using cached https://files.pythonhosted.org/packages/d4/a4/1face47e65118d7c52726dfa305410a96bc4a0c6f3f99c90bc7104aebf21/python-olm-3.1.3.tar.gz
Requirement already satisfied: cffi>=1.0.0 in c:\users\me\appdata\local\programs\python\python38\lib\site-packages (from python-olm) (1.13.2)
Requirement already satisfied: future in c:\users\me\appdata\local\programs\python\python38\lib\site-packages (from python-olm) (0.18.2)
Requirement already satisfied: pycparser in c:\users\me\appdata\local\programs\python\python38\lib\site-packages (from cffi>=1.0.0->python-olm) (2.19)
Building wheels for collected packages: python-olm
Building wheel for python-olm (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\me\appdata\local\programs\python\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"'; __file__='"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\me\AppData\Local\Temp\pip-wheel-vv1l60m8' --python-tag cp38
cwd: C:\Users\me\AppData\Local\Temp\pip-install-1myzii8l\python-olm\
Complete output (30 lines):
'make' is not recognized as an internal or external command,
operable program or batch file.
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
creating build\lib.win-amd64-3.8\olm
copying olm\account.py -> build\lib.win-amd64-3.8\olm
copying olm\group_session.py -> build\lib.win-amd64-3.8\olm
copying olm\pk.py -> build\lib.win-amd64-3.8\olm
copying olm\sas.py -> build\lib.win-amd64-3.8\olm
copying olm\session.py -> build\lib.win-amd64-3.8\olm
copying olm\utility.py -> build\lib.win-amd64-3.8\olm
copying olm\_compat.py -> build\lib.win-amd64-3.8\olm
copying olm\_finalize.py -> build\lib.win-amd64-3.8\olm
copying olm\__init__.py -> build\lib.win-amd64-3.8\olm
copying olm\__version__.py -> build\lib.win-amd64-3.8\olm
running build_ext
generating cffi module 'build\\temp.win-amd64-3.8\\Release\\_libolm.c'
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
building '_libolm' extension
creating build\temp.win-amd64-3.8\Release\build
creating build\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.24.28314\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\users\me\appdata\local\programs\python\python38\include -Ic:\users\me\appdata\local\programs\python\python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.24.28314\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcbuild\temp.win-amd64-3.8\Release\_libolm.c /Fobuild\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8\Release\_libolm.obj -I../include
_libolm.c
build\temp.win-amd64-3.8\Release\_libolm.c(493): fatal error C1083: Cannot open include file: 'olm/olm.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.24.28314\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Failed building wheel for python-olm
Running setup.py clean for python-olm
Failed to build python-olm
Installing collected packages: python-olm
Running setup.py install for python-olm ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\me\appdata\local\programs\python\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"'; __file__='"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\me\AppData\Local\Temp\pip-record-kcobwouw\install-record.txt' --single-version-externally-managed --compile
cwd: C:\Users\me\AppData\Local\Temp\pip-install-1myzii8l\python-olm\
Complete output (30 lines):
'make' is not recognized as an internal or external command,
operable program or batch file.
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
creating build\lib.win-amd64-3.8\olm
copying olm\account.py -> build\lib.win-amd64-3.8\olm
copying olm\group_session.py -> build\lib.win-amd64-3.8\olm
copying olm\pk.py -> build\lib.win-amd64-3.8\olm
copying olm\sas.py -> build\lib.win-amd64-3.8\olm
copying olm\session.py -> build\lib.win-amd64-3.8\olm
copying olm\utility.py -> build\lib.win-amd64-3.8\olm
copying olm\_compat.py -> build\lib.win-amd64-3.8\olm
copying olm\_finalize.py -> build\lib.win-amd64-3.8\olm
copying olm\__init__.py -> build\lib.win-amd64-3.8\olm
copying olm\__version__.py -> build\lib.win-amd64-3.8\olm
running build_ext
generating cffi module 'build\\temp.win-amd64-3.8\\Release\\_libolm.c'
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
building '_libolm' extension
creating build\temp.win-amd64-3.8\Release\build
creating build\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.24.28314\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\users\me\appdata\local\programs\python\python38\include -Ic:\users\me\appdata\local\programs\python\python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.24.28314\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcbuild\temp.win-amd64-3.8\Release\_libolm.c /Fobuild\temp.win-amd64-3.8\Release\build\temp.win-amd64-3.8\Release\_libolm.obj -I../include
_libolm.c
build\temp.win-amd64-3.8\Release\_libolm.c(493): fatal error C1083: Cannot open include file: 'olm/olm.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.24.28314\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\me\appdata\local\programs\python\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"'; __file__='"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-1myzii8l\\python-olm\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\me\AppData\Local\Temp\pip-record-kcobwouw\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.
The Python bindings require the C lib and C headers to be installed beforehand. @turt2live could you lend a hand in Windows land?
I believe the error about the make file is safely ignored even though it shouldn't be there, this should be fixed by next release.
Thanks for the report.
The Python bindings require the C lib and C headers to be installed beforehand
Compiling on Windows is a terrible user experience for people who develope in Python and have no experience with C/C++. The knowledge isn't there, the build tools aren't there, and where are we supposed to get C libs/headers or place them? For example, I had to install the VS C++ build tools, which already required me to install a 4 GB package of Visual Studio on my computer only to compile a small module (which then still didn't work, as shown above).
The best experience for any Windows user is, when they find a binary wheel uploaded to Pypi.
This system of conflating build-install is slow, hard to maintain, and hinders innovation in both build systems and installers.
Wheel attempts to remedy these problems by providing a simpler interface between the build system and the installer. The wheel binary package format frees installers from having to know about the build system, saves time by amortizing compile time over many installations, and removes the need to install a build system in the target environment.
https://www.python.org/dev/peps/pep-0427/
Would be great if that could happen when you release the next version. 💚
Compiling on Windows is a terrible user experience for people who develope in Python ...
yes it is, particularly when you don't have a windows machine to build on... which is a big reason we don't ship windows binaries.
We might want to let CI build statically linked wheels using something like cibuildwheel.
CI would in this case push releases to PyPi as well, but that requires CI to be set up and I forgot what our stance on the legal side of things is with regards to this.
Anybody got it compiled?
It builds under mingw-w64 with a bit of effort, if someone cleans up the build files to handle the environment the process would be easy for most Windows users. Here's a how to get it working in the meantime:
Setup the mingw-w64 environment with updated dependencies and make sure you are in the 64-bit MINGW shell (not MSYS). Also make sure to use mingw's pip.
pacman -Syu mingw-w64-x86_64-python-pip libcrypt-devel libffi-devel
/mingw64/bin/pip install --upgrade pip
/mingw64/bin/pip install --upgrade setuptools
/mingw64/bin/pip install --upgrade wheel
/mingw64/bin/pip install --upgrade cffi
Get the latest olm source and build the static library instead of the default dynamic library. You also need to explicitly tell cmake to use the standard c++ library (libstdc++.a) since it doesn't assume this on it's own.
git clone https://gitlab.matrix.org/matrix-org/olm.git
cd olm
cmake . -Bbuild -DBUILD_SHARED_LIBS=NO -DCMAKE_CXX_STANDARD_LIBRARIES="-lstdc++"
cmake --build build
Once the build is done you should have the olm static library (libolm.a) in your build directory. Test it by running ctest while inside the build/tests directory and all tests should pass.
pushd build/tests
ctest .
popd
The next step is to build the Python wheel from the python directory.
cd python
python setup.py bdist_wheel
This will fail because the generated compile command for _libolm-cpython-38.dll misses the libstdc++.a requirement and fails to put the library paths (-L arguments) before the libraries themselves (-l arguments). You also need to add a -static argument to build the shared library without other shared library dependencies. It's good practice (may not be necessary) to have it reference the Python shared library where you will use the wheel instead of mingw's Python. On my system that was C:\Python38\python38.dll and you can specify that by replacing -LC:/msys64/mingw64/lib/python3.8/config-3.8 and -lpython3.8 with the arguments -L/c/python38 and -l:python38.dll (note the colon is needed to specify the exact file name). The fixed compile command looked like this on my system.
/c/msys64/mingw64/bin/x86_64-w64-mingw32-gcc.exe -shared -static -Wl,--enable-auto-image-base -pipe -s build/temp.mingw-3.8/build/temp.mingw-3.8/_libolm.o build/temp.mingw-3.8/build/temp.mingw-3.8/_libolm-cpython-38.def -LC:/msys64/mingw64/lib -L../build -L/c/python38 -lolm -l:python38.dll -lm -lversion -lshlwapi -lstdc++ -o build/lib.mingw-3.8/_libolm-cpython-38.dll
After re-running the compile command you can continue building the Python wheel.
python setup.py bdist_wheel
You should now have a completed wheel (python_olm-3.1.4-cp38-cp38-mingw.whl) in your dist directory, but it specifies mingw as the platform instead of win_amd64 and it's expecting the shared library to be _libolm.dll instead of _libolm-cpython-38.dll. Both of these issues can be fixed by re-packaging the wheel.
cd dist
wheel unpack python_olm-3.1.4-cp38-cp38-mingw.whl
mv python_olm-3.1.4/_libolm-cpython-38.dll python_olm-3.1.4/_libolm.pyd
perl -i -pe 's/cp38-cp38-mingw/cp38-cp38-win_amd64/' python_olm-3.1.4/python_olm-3.1.4.dist-info/WHEEL
wheel pack python_olm-3.1.4
If all of that worked you'll end up with a wheel called python_olm-3.1.4-cp38-cp38-win_amd64.whl that you can install like any other wheel.
@dirkbike Would it be possible to upload the wheel for cp38 and win64? This issue still seems to be unsolved.
This issue is still not resolved. Apart from the method @dirkbike mentioned, is there an easier way to install python-olm on windows?
It would be great if we'd get the .whl file from the aforementioned method.
Thanks
Why not use wsl2? It works fine for me.
Please add prebuilt libs or Windows onto pypi.