eclipse-zenoh-flow/zenoh-flow

[Bug] Not work on OpenCV+Qt

YuanYuYuan opened this issue · 4 comments

YuanYuYuan commented

Describe the bug

I encountered the following bug when executing this simple video pipeline.

QObject::startTimer: Timers cannot be started from another thread

In short, we may have a problem if we try to use multi-threading OpenCV with the Qt backend. Since Qt is also a common GUI backend besides GTK, I would suggest some alternative solutions to the mandatory async Source/Sink trait in zenoh-flow in this case.

To reproduce

Please visit here.

System info

I found this issue because I use Arch on daily use. Here I summarize my observations.

  • Arch Linux (latest)
    • Default package: OpenCV 4.6.0 + Qt (failed)
  • Ubuntu 20.04
    • Default package: OpenCV 4.2.0 + GTK (passed)
    • Manual build: OpenCV 4.6.0 + Qt (failed)
gabrik commented

Hi, @YuanYuYuan I think the problem will always happen even if we change the interface of Zenoh Flow nodes. (e.g., provide sync Traits).

Let me go into more in detail:
As we are using async_std we have very little control over which thread the code is actually executing, and most UI frameworks require that GUI operations happen on the same thread (or the main one in the case of macOS).

I would also point out that Zenoh Flow should not be used to spawn GUIs, as those have specific requirements on threading, access to the window server, etc.
Zenoh Flow is designed to run as a daemon, and therefore without access to the window server, screens, and all the required UI-related stuff.

On this line, we should remove all the examples that make use of UI as they are misleading.

YuanYuYuan commented

@gabrik , thanks for your information! But I'm wondering if zenoh-flow is running as a daemon without GUIs, then how can we display those in-memory OpenCV Mat originally captured from the source node? Does it necessarily need message passing over TCP?

YuanYuYuan commented

BTW, I tried to write a sample to manipulate OpenCV in our usual way with zenoh-flow here. It can work fast since we spawn a dedicating thread for it. Is there any similar idea in zenoh-flow-examples?

J-Loudet commented

I'm wondering if zenoh-flow is running as a daemon without GUIs, then how can we display those in-memory OpenCV Mat originally captured from the source node? Does it necessarily need message passing over TCP?

Message passing over TCP is, indeed, a possibility (definitely not the only one). What’s important to remember is that the display has to be done "outside" of Zenoh-Flow — as, as mentioned by @gabrik, we do not control on which thread the nodes are executed, which is a prerequisite for most UI frameworks.

BTW, I tried to write a sample to manipulate OpenCV in our usual way with zenoh-flow here. It can work fast since we spawn a dedicating thread for it. Is there any similar idea in zenoh-flow-examples?

In your example you used the programmatic API to interact with Zenoh-Flow. This programmatic API is not as polished as the one relying on the daemon (we might want to remove it at some point) albeit convenient to run tests and measure performance.
Considering this, we did not provide any example that leverage it and thus nothing similar. This could change in the near future as you raised fair points.