bug: Napari stylesheet paths messed up in mkdocs plugin markdown-exec

Description of the bug

In /docs/snippets/ I run the following during MkDocs serve:

from napari.qt import get_stylesheet
from plantseg.[some module] import widget  # a magicgui widget

widget.native.setStyleSheet(get_stylesheet("dark"))

WARNING: Cannot open file '/yu/repositories/plant-seg/theme_light:/minus_50.svg', because: No such file or directory
WARNING:vispy:Cannot open file '/yu/repositories/plant-seg/theme_light:/minus_50.svg', because: No such file or directory
WARNING: Cannot open file '/yu/repositories/plant-seg/theme_dark:/drop_down_50.svg', because: No such file or directory
WARNING:vispy:Cannot open file '/yu/repositories/plant-seg/theme_dark:/drop_down_50.svg', because: No such file or directory
WARNING: Cannot open file '/yu/repositories/plant-seg/theme_dark:/plus_50.svg', because: No such file or directory
WARNING:vispy:Cannot open file '/yu/repositories/plant-seg/theme_dark:/plus_50.svg', because: No such file or directory

To Reproduce

But I don't really have time today to finish the rest of this issue, because a "minimal" example involves creating a widget that uses "theme_light:/minus_50.svg". It would be great if you can think of an easy solution with the description above, otherwise I'll come back to this tomorrow.

Environment information

python -m markdown_exec.debug  # | xclip -selection clipboard

System: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.28
Python: cpython 3.11.9 (/yu/miniconda3/envs/plant-seg/bin/python)
Environment variables:
Installed packages:
- markdown-exec v1.8.3

Hi @qin-yu, thanks for the report :)

I never used napari or plantseg, so I will not have time to find out what the issue exactly is. My intuition is that it's a "current working directory" issue, if you want to look into that.

By the way markdown-exec exports an environment variable that points to the directory containing the MkDocs configuration file when used as a MkDocs plugin. I'll try to send you a docs link asap.

My intuition is that it's a "current working directory" issue, if you want to look into that.

Thanks Timothée! Exactly, this is also what I thought. I think markdown-exec might have prepend a current working directory to the CSS I got from napari.qt.get_stylesheet('dark'), which looks like:

QComboBox::down-arrow {
   image: url("theme_dark:/drop_down_50.svg");
   width: 14px;
   height: 14px;
}

Current debugging progress

So if I run the same code outside mkdocs context, then there would be no warnings, which means some parts of mkdocs changes the prefix/cwd for paths.

I wasn't expecting you to debug for me, but just hoping maybe you knew how paths for python execution are manipulated in a mkdocs-build context.

One thing I tried was to print(os.environ) right before widget_rescaling.native.setStyleSheet(get_stylesheet('dark')) in both cases and diff it. PWDs are the same but there is one difference:

'MKDOCS_CONFIG_DIR': '/yu/repositories/plant-seg',

Then I go to markdown-exec repo and see how MKDOCS_CONFIG_DIR is used, and found 3 Python files.

But I didn't pay for insider, so can't be this one:

markdown-exec/scripts/insiders.py

Line 125 in 97de061

project_dir = os.getenv("MKDOCS_CONFIG_DIR", ".")

and this one is just printing credit:

markdown-exec/scripts/gen_credits.py

Line 24 in 97de061

project_dir = Path(os.getenv("MKDOCS_CONFIG_DIR", "."))

So it might be this one:

markdown-exec/src/markdown_exec/mkdocs_plugin.py

Line 79 in 97de061

self.mkdocs_config_dir = os.getenv("MKDOCS_CONFIG_DIR")

So I looked into that because the filename and function seem relevant, but thought maybe I should ask you before I dig deeper.

By the way markdown-exec exports an environment variable that points to the directory containing the MkDocs configuration file when used as a MkDocs plugin. I'll try to send you a docs link asap.

Oh cool, thanks @pawamoy! I have a feeling this might be exactly what I was looking for

Yes, you were on the right path! The purpose of this env var is to use it to change the current working directory or build paths relative to your docs dir when needed. In your case I'm not sure exactly what's best, but it's always possible to use os.chdir in last resort.

Also the env var exists because docs can be built from any directory on the file system with Mkdocs' -f option, so we needed a way to stay relative to the docs dir for markdown-exec snippets.

it's always possible to use os.chdir in last resort.

I don't really know where should I os.chdir to. Maybe I should ask napari people where do I actually find this file url("theme_dark:/drop_down_50.svg"), because it seems to be dynamically generated? I'm not familiar with CSS (.qss in this case), and drop_down_50.svg only appears once in this .qss file or it's duplicate across GitHub. I don't find a Resource Collection File .qrc file anywhere either..

Hi @pawamoy, it might be really hard to figure out on the napari side: https://forum.image.sc/t/where-is-url-theme-id-drop-down-50-svg-from-napari/.

Is this file https://github.com/pawamoy/markdown-exec/blob/main/src/markdown_exec/mkdocs_plugin.py all I need to know or there are other parts I need to pay attention too, in order to skip prepending current working dir / mkdocs yaml dir to napari css / qt rendering?

I'm not even sure where it happens on napari/qt side, but it's one line so I want to do:

undo_changes_to_env_or_paths()
widget.native.setStyleSheet(get_stylesheet("dark")) 
redo_changes_to_env_or_paths()

@qin-yu markdown-exec actually doesn't do anything with the current working directory. The env var we provide is here just in case one of the user's snippets actually requires to move to a path relative to the docs (or build such a relative path to pass it to a downstream library's API). The file you mention is indeed the only relevant files (the other two are just for our own docs).

Also, since you get WARNINGS and not ERRORS, maybe the solution is just to filter out these warnings, if they actually don't cause any error, so that your strict build passes. See https://docs.python.org/3/library/warnings.html#warning-filter.

I think markdown-exec might have prepend a current working directory to the CSS I got from napari.qt.get_stylesheet('dark')

Please try to write the CSS contents you obtain from get_stylesheet while buildling your docs, and compare it to the CSS you get when running outside of MkDocs. I don't know how this get_stylesheet function is written, but there's no reason executing it through markdown-exec would change the URLs in the CSS.

Don't you get the same warnings anyway when running outside of MkDocs? Can you complete your initial snippet?

from napari.qt import get_stylesheet
from plantseg.[some module] import widget  # a magicgui widget

widget.native.setStyleSheet(get_stylesheet("dark"))

What's [some module] here? I suspect your code always generate these warnings anyway, and maybe you didn't see them before because Python doesn't show them by default.

Try running your code outside of MkDocs with python -Walways so that all warnings are shown. If you see them, then markdown-exec has nothing to do with this and we can close 🙂

Please try to write the CSS contents you obtain from get_stylesheet while buildling your docs, and compare it to the CSS you get when running outside of MkDocs. I don't know how this get_stylesheet function is written, but there's no reason executing it through markdown-exec would change the URLs in the CSS.

I think the problem is from widget.native.setStyleSheet(), which is a Qt function so I probably should look into this, but PyQt doesn't seem to be on GitHub... I asked here (but in Mandarin because it seems like a Chinese community repo): PyQt5/PyQt#188

Don't you get the same warnings anyway when running outside of MkDocs? Can you complete your initial snippet?

The test I ran outside MkDocs is:

from plantseg.viewer.widget.dataprocessing import widget_rescaling, RescaleModes
from napari.qt import get_stylesheet
import os

# print(os.environ)  # show all environment variables and their values.

widget_rescaling.mode.value = RescaleModes.FROM_FACTOR

widget_rescaling.show()
widget_rescaling.native.setStyleSheet(get_stylesheet('dark'))

Try running your code outside of MkDocs with python -Walways so that all warnings are shown. If you see them, then markdown-exec has nothing to do with this and we can close 🙂

PlantSeg is by default configured to show warnings but I also tried what you recommend:

(plant-seg) $ python -Walways /yu/repositories/plant-seg/test_style.py
/yu/miniconda3/envs/plant-seg/lib/python3.11/site-packages/jupyter_client/connect.py:22: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
given by the platformdirs library.  To remove this warning and
see the appropriate new directories, set the environment variable
`JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
The use of platformdirs will be the default in `jupyter_core` v6
  from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write
WARNING: Qt: Session management error: Authentication Rejected, reason : None of the authentication protocols specified are supported and host-based authentication failed
2024-06-05 14:26:57,672 [MainThread] INFO PlantSeg Zoo - Fetching BioImage.IO Model Zoo collection from https://raw.githubusercontent.com/bioimage-io/collection-bioimage-io/gh-pages/collection.json
2024-06-05 14:26:57,677 [MainThread] INFO PlantSeg Zoo - Loaded model from PlantSeg zoo: generic_confocal_3D_unet
2024-06-05 14:26:57,732 [MainThread] INFO PlantSeg Zoo - Loaded model from user specified weights: /home/qyu/.plantseg_models/generic_confocal_3D_unet/best_checkpoint.pytorch

I don't know if the program terminates before the warning is printed if at all.

Hi @pawamoy thank you so much for your time and quick replies. I think I solved the problem, and replied to https://forum.image.sc/t/where-is-url-theme-id-drop-down-50-svg-from-napari/ already. I'm really sorry to drag you to this rabbit hole with me, but here is what might have happened:

Solution

To resolve the issue, simply add the following line to set the search path for Qt:

PyQt5.QtCore.QDir.addSearchPath(f'theme_{name}', str(_theme_path(name)))

What it means is, your markdown-exec never messed up the paths, but napari never initialised it's QApplication singleton when I just want to document the screenshots of single widgets on its own.

Cause

The reason why Qt inside magicgui widgets couldn’t locate the SVG file built by Napari is that QApplication never initializes in the context of an MkDocs build using markdown-exec.

Under normal usage, napari._qt.qt_event_loop.get_app() gets the singleton Napari app, and the line PyQt5.QtCore.QDir.addSearchPath(f'theme_{name}', str(_theme_path(name))) within this context allows Qt to recognize the temporary path for all SVGs, such as:

/home/qyu/.cache/napari/plant-seg_1bbf9c2c85c974192c72e305681854d064a243cf/_themes/dark

To verify this, you can simply instantiate a Napari viewer with viewer = napari.Viewer() before calling .setStyleSheet(), which also resolves the problem.

Thank you so much again!

I tried to simplify the problem into

from napari.qt import get_stylesheet
from plantseg.[some module] import widget  # a magicgui widget

widget.native.setStyleSheet(get_stylesheet("dark"))

and this was absolutely wrong, because in reality I run widget.native.setStyleSheet(get_stylesheet("dark")) in docs/snippets/render.py for each widget imported in files like docs/snippets/widgets/document_this_widget_*.py, i.e. importing the widget in the docs/snippets/render.py should fix the problem.

OK, let me know how it goes :)

Hey @pawamoy I made three new comments after your last reply, maybe you only saw the first and the last one. GitHub/Chrome doesn't always show all comments and even hides the new ones on refresh. The problem is solved!

To resolve the issue, simply add the following line to set the search path for Qt:
PyQt5.QtCore.QDir.addSearchPath(f'theme_{name}', str(_theme_path(name)))

But the solution in the last comment doesn't really work, just tested this and failed:

importing the widget in the docs/snippets/render.py should fix the problem.

You may close this issue if you don't think anything needs to be done, or I can write a few lines in your documentation if you think this would benefit the community. This is what I use your repo for:

https://kreshuklab.github.io/plant-seg/chapters/plantseg_interactive_napari/data_processing/

PlantSeg Docs Screenshots

PlantSeg Docs

Oh, yes indeed, I only saw the last comment (using GitHub mobile). Thanks for the heads up! Glad to hear your issue is resolved, I will therefore close this :)