Using opencv + python in feedstocks now requires more opengl drivers
Opened this issue · 14 comments
This is only a message for the linux builds. All other builds should be unaffected by this change.
Comment:
Since we added qt6-main as a dependency of opencv by "default", it is causing errors for downstream projects. I'm not sure if we can make these libraries "optional" at runtime for qt6 to help downstream projects "build" what they need without adding the OpenGL stuff.
the following packages should be added to the yum_requirements.txt file
mesa-libGL
mesa-libEGL
libglvnd-egl
libglvnd-glx
libglvnd-opengl
After adding these lines, one must rerender the recipe.
First reported in: #400 by @paugier (thank you for being patient)
Presently, I would suggest that in the test section, users that are facing this issue add:
test:
requires:
- libopencv *=headless*
Please do not specify a build string constraint for runtime requirements. It will override the run exports and restrict how users can use opencv in their system. Adding a requirement to the testing section is safe since it tests against the most reduced set of functionality.
For Ubuntu users and WSL+Ubuntu users:
apt-get install libglapi-mesa libegl-mesa0 libegl1 libopengl0 libgl1-mesa-glx
cc: @conda-forge/qt-main
@hmaarrfk thanks for raising this issue, this has been exactly our problem for several weeks in https://github.com/conda-forge/ultralytics-feedstock with about a dozen patch updates failing to deploy.
Adding this line to tests resolves the build issues, but what about users installing our package without seeing this issue? Will their opencv installs fail without these additional opengl dependencies?
Yeah, unfortunately i haven't had the chance to analyze this in detail.
I have 2 answer to this:
- The users deploying with docker + conda on the web should in general be able to benefit from this by adding the constraint to their build environments. The change that introduced the qt6 build also introduced a headless build that should start to avoid the qt5 inclusion (which was mandatory last week) and no in significantly small. I'm trying to add a
noqtpackage so that users of headless machines can "ask conda" to forcibly not install qt5 or qt6 (saving space) but I would like more feedback. - "Beginners" will likely be using Ubuntu or some other graphical Linux environment. So this shouldn't be noticeable for them.
If the above isn't really true, I'm happy to revisit (potentially marking linux builds with opencv+qt6 as broken, but this is somewhat drastic).
We could, maybe reconsider making the headless version the default one. But I think that would benefit "experts" more than beginners, which is somewhat against what brought me to conda in the first place.
Ultimately, this #293 (comment) comment is what I would like to do, but you know 3 years later and limited time has made this a nice to have.
I'm very interested in hearing user stories, without them, its hard to choose the right path.
As a followup, from a historical prospective, it seems that things like mesa-libGl were added in the past to support the Qt5 graphical path in cv2
https://github.com/conda-forge/ultralytics-feedstock/blob/main/recipe/yum_requirements.txt
What about Windows? I now have the same type of issue on Windows (to build a new version of Fluidimage for conda-forge, conda-forge/fluidimage-feedstock#21)
import: 'cv2'
Traceback (most recent call last):
File "D:\bld\fluidimage_1711573070427\test_tmp\run_test.py", line 2, in <module>
import cv2
ImportError: DLL load failed while importing cv2: The specified module could not be found.
https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=904591&view=results
There is no headless version for Windows, so I cannot use this. I also tried to add the yum_requirements.txt but it does not help (for Windows).
What about Windows? I now have the same type of issue on Windows (to build a new version of Fluidimage for conda-forge, conda-forge/fluidimage-feedstock#21)
import: 'cv2' Traceback (most recent call last): File "D:\bld\fluidimage_1711573070427\test_tmp\run_test.py", line 2, in <module> import cv2 ImportError: DLL load failed while importing cv2: The specified module could not be found.https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=904591&view=results
There is no headless version for Windows, so I cannot use this. I also tried to add the
yum_requirements.txtbut it does not help (for Windows).
I guess this is a different problem, tracked in conda-forge/graphite2-feedstock#14 . Quick workaround: conda install graphite2==1.3.13=1000
This is breaking CI systems probably all over the place. I maintain a scientific piece of software and it's not as easy a fix for me because we don't want opencv to be headless (we use it for both maths and visualisations) but we also don't necessarily have the ability to install opengl on github test systems (or other places).
I'm not sure there is a good solution here, unless we can get the Qt developers to stop depending on OpenGL.
This is breaking CI systems probably all over the place.
Yes I agree, though maybe the qt6 hard dependency on opengl is best discussed in: https://github.com/conda-forge/qt-main-feedstock/
we don't want opencv to be headless
To retain the old behavior of opencv on linux (requiring qt5) you would do
- libopencv *=qt5*
However, I have to say that qt5 is somewhat hard to maintain. I've had to backport a difficult patch conda-forge/qt-main-feedstock#244 to enable modernizing it. So the qt6 transition is meant to help application like yours in the long run....
Will add the ubuntu dependency list to the top comment to help Ubuntu and WSL ubuntu users.
I just ran into the same issue. I have updated yum_requirements.txt, but still got the ImportError: libEGL.so.1: cannot open shared object file: No such file or directory. Any advice?
https://github.com/conda-forge/segment-geospatial-feedstock/pull/26/files
mesa-libGL
mesa-libEGL
libglvnd-egl
libglvnd-glx
libglvnd-opengl
Adding the following resolves the issue.
test:
requires:
- libopencv *=headless* # [linux]
- ```I just ran into the same issue. I have updated
yum_requirements.txt, but still got theImportError: libEGL.so.1: cannot open shared object file: No such file or directory. Any advice?https://github.com/conda-forge/segment-geospatial-feedstock/pull/26/files
mesa-libGL mesa-libEGL libglvnd-egl libglvnd-glx libglvnd-opengl
You must rerender after this addition. I added this in the instructions.
Hello,
I am encountering this issue too: conda-forge/detectree-feedstock#4
However, I am only using opencv to peform convolutions, i.e., the cv2.filter2D method, so I suppose that I do not need all the qt/graphics dependencies. Therefore, if I understand correctly from #293, in order to keep the conda package as small as possible, I could change opencv for opencv-headless in the run requirements of my recipe and then I would not need to add the yum_requirements.txt file? Does this make sense?
Thank you. Best,
Martí
thank you for asking the question.
Do not change the run requirements as it would conflict with other packages that may require opencv's GUI freature in your end user's final environment.
Rather simply add a headless constraint to your test section. I made a suggestion to your feedstock.
Thank you @hmaarrfk! I applied your suggestion, tests passed and I could already merge the PR 🚀