Windows Bug Version Dependencies???
heeler opened this issue · 5 comments
Doug & Paul encountered this bug
Microsoft Windows [Version 10.0.17134.950]
(c) 2018 Microsoft Corporation. All rights reserved.
C:\Users\Doug>python C:\dev\pylibczi-testing\readmultisceneczi2.py
D:\czi multiscene\20190807_RS013_MJM01_Doug_1_000.czi
Traceback (most recent call last):
File "C:\dev\pylibczi-testing\readmultisceneczi2.py", line 6, in <module>
czi = CziFile("D:\\czi multiscene\\20190807_RS013_MJM01_Doug_1_000.czi")
File "C:\Program Files\Python37\lib\site-packages\aicspylibczi\CziFile.py", line 71, in __init__
import _pylibczi
ImportError: DLL load failed: The specified module could not be found.
C:\Users\Doug>dir "C:\Program Files\Python37\Lib\site-packages"
Volume in drive C is OS
Volume Serial Number is 75C0-2803
Directory of C:\Program Files\Python37\Lib\site-packages
11/12/2019 08:52 AM
.11/12/2019 08:52 AM ..
11/12/2019 08:52 AM aicspylibczi
11/12/2019 08:52 AM aicspylibczi-2.1.0.dist-info
07/03/2019 09:50 AM cffi
07/03/2019 09:50 AM cffi-1.12.3.dist-info
08/06/2019 12:43 PM cmake
08/06/2019 12:43 PM cmake-3.14.4.dist-info
07/03/2019 09:49 AM cv2
08/06/2019 12:42 PM cycler-0.10.0.dist-info
08/06/2019 12:41 PM 15,959 cycler.py
07/03/2019 09:49 AM czifile
07/03/2019 09:49 AM czifile-2019.7.2.dist-info
07/03/2019 09:51 AM dateutil
08/06/2019 12:41 PM decorator-4.4.0.dist-info
08/06/2019 12:41 PM 17,244 decorator.py
11/12/2019 08:50 AM 0 easy-install.pth
07/03/2019 09:47 AM 126 easy_install.py
07/03/2019 09:49 AM imagecodecs
07/03/2019 09:49 AM imagecodecs-2019.5.22.dist-info
08/06/2019 12:41 PM imageio
08/06/2019 12:41 PM imageio-2.5.0.dist-info
08/06/2019 12:42 PM kiwisolver-1.1.0.dist-info
08/06/2019 12:41 PM 143,872 kiwisolver.cp37-win_amd64.pyd
07/03/2019 11:10 AM lxml
07/03/2019 11:10 AM lxml-4.3.4.dist-info
08/06/2019 12:42 PM matplotlib
08/06/2019 12:41 PM 570 matplotlib-3.1.1-py3.7-nspkg.pth
08/06/2019 12:42 PM matplotlib-3.1.1.dist-info
08/06/2019 12:42 PM mpl_toolkits
08/06/2019 12:41 PM networkx
08/06/2019 12:41 PM networkx-2.3-py3.7.egg-info
07/03/2019 09:49 AM numpy
07/03/2019 09:49 AM numpy-1.16.4.dist-info
07/03/2019 09:49 AM opencv_python-4.1.0.25.dist-info
07/03/2019 09:52 AM pandas
07/03/2019 09:52 AM pandas-0.24.2.dist-info
08/06/2019 12:41 PM PIL
08/06/2019 12:41 PM Pillow-6.1.0.dist-info
11/12/2019 08:49 AM pip
11/12/2019 08:49 AM pip-19.3.1.dist-info
07/03/2019 09:47 AM pkg_resources
07/03/2019 09:50 AM pycparser
07/03/2019 09:50 AM pycparser-2.19.dist-info
08/06/2019 12:41 PM 90 pylab.py
08/06/2019 12:42 PM pyparsing-2.4.2.dist-info
08/06/2019 12:41 PM 263,439 pyparsing.py
07/03/2019 09:51 AM python_dateutil-2.8.0.dist-info
07/03/2019 09:52 AM pytz
07/03/2019 09:52 AM pytz-2019.1.dist-info
07/03/2019 09:50 AM pyvips
07/03/2019 09:50 AM pyvips-2.1.8-py3.7.egg-info
08/06/2019 12:41 PM PyWavelets-1.0.3.dist-info
08/06/2019 12:41 PM pywt
03/25/2019 08:21 PM 121 README.txt
08/06/2019 12:42 PM scikit_image-0.15.0.dist-info
08/06/2019 12:41 PM scipy
08/06/2019 12:41 PM scipy-1.3.0.dist-info
07/03/2019 09:47 AM setuptools
07/03/2019 09:47 AM setuptools-40.8.0.dist-info
07/03/2019 09:51 AM six-1.12.0.dist-info
07/03/2019 09:51 AM 32,452 six.py
08/06/2019 12:42 PM skimage
07/03/2019 09:49 AM tifffile
07/03/2019 09:49 AM tifffile-2019.7.2.dist-info
07/03/2019 09:50 AM 174,592 _cffi_backend.cp37-win_amd64.pyd
11/12/2019 08:52 AM 836,096 _pylibczi.cp37-win_amd64.pyd
08/06/2019 12:42 PM pycache
12 File(s) 1,484,561 bytes
56 Dir(s) 1,684,420,505,600 bytes free
C:\Users\Doug>
Here is the script readmultisceneczi2.py:
from PIL import Image
import aicspylibczi as pylibczi
from aicspylibczi import CziFile
czi = CziFile("D:\\czi multiscene\\20190807_RS013_MJM01_Doug_1_000.czi")
print(czi)
im = czi.read_mosaic(scale_factor=0.1, C=1)
print(im)
img = Image.fromarray(im)
print(img)I'm having trouble reproducing this. I'm trying to get my hands on a windows box to mess around with.
@elhuhdron commented:
I tried a dumpbin on windows of the .pyd binary, and it has dependencies on the MSVC runtimes, which is expected. I have a feeling the versioning on the runtimes is the problem. I'm a bit confused on the build matrix now; which compiler is used for windows and with runtime version? From a quick search it appears that Visual Studio 2015 and 2017 versions are not compatible, so if we are building the binary against 2015, this might be the problem. Windows won't even let me install the 2015 runtime version if 2017 is already installed. I also found a couple threads indicating it might be possible with a linker flag to statically link the runtimes, which is probably most ideal as hopefully it could avoid this issue all together:
"You can use the /MT switch to statically link to CRT"
I was able to resolve this issue on my system by updating the MSVC runtimes from here.
I already had the 2017 runtimes installed, but apparently this is not sufficient for binaries built with 2019 even though MS describes it as a "combined 2015-2019 runtime".
I think it would be better if we can find a method that statically links the runtimes, as it would prevent this issue going forward. Otherwise we should add to the README that for windows the runtimes need to be updated to the latest versions.
I think it would be better if we can find a method that statically links the runtimes, as it would prevent this issue going forward.
That's not a good idea since there is a limit on how many of such C extensions/DLLs can be loaded into a process. See https://bugs.python.org/issue25027
Also the GPL license doesn't allow this if you want to distribute the binaries. See https://www.gnu.org/licenses/gpl-faq.en.html#WindowsRuntimeAndGPL. Thanks to Zeiss for choosing GPL...
@elhuhdron Thanks you for figuring out it was the runtime!
@cgohlke I really appreciate the comment on the GPL, I would not have caught that subtlety.
My thought as to how to best fix this generally is to change my GitHub actions build environment for PyPI publication to Windows Server 2016 and build against MSVC 2017 as this is a backward compatibility bug between MSVC 2019 and any windows system with older runtimes. The alternative is as @elhuhdron suggests putting in a link to the runtime and guiding people to install from there. Which I will mention as well. Comments on these ideas welcome.