samhocevar/portable-file-dialogs

Dialog error inside Flatpak container

megumumpkin opened this issue · 12 comments

So I'm developing a software using the game engine WickedEngine, this engine uses your library for opening dialogs cross platform between Windows and Linux, for my specific use case I'm using the Linux version of the engine and develop the software using Flatpak container for publishing and development (debugging within Flatpak environment).

The thing is, within the Flatpak environment the file dialog is skipped entirely, this slows down development process as I have to skip debugging for anything that requires file saving/loading behavior (which is not ideal). The container runs GTK I think, and it has zenity inside it.

So I decided to test and debug myself on to your library's dialog process, after stepping through the instruction process I got to the source of the problem, which is the part of code inside the thread execution of the executor class, specifically the start_process() function.

Down below is the part of code that I had mentioned.

inline void internal::executor::start_process(std::vector<std::string> const &command)
{
    stop();
    m_stdout.clear();
    m_exit_code = -1;

    int in[2], out[2];
    if (pipe(in) != 0 || pipe(out) != 0)
        return;

    m_pid = fork();
    if (m_pid < 0)
        return;

    close(in[m_pid ? 0 : 1]);
    close(out[m_pid ? 1 : 0]);

    //This part is skipped---------------------------------------------------------
    if (m_pid == 0)
    {
        dup2(in[0], STDIN_FILENO);
        dup2(out[1], STDOUT_FILENO);

        // Ignore stderr so that it doesn’t pollute the console (e.g. GTK+ errors from zenity)
        int fd = open("/dev/null", O_WRONLY);
        dup2(fd, STDERR_FILENO);
        close(fd);

        std::vector<char *> args;
        std::transform(command.cbegin(), command.cend(), std::back_inserter(args),
                       [](std::string const &s) { return const_cast<char *>(s.c_str()); });
        args.push_back(nullptr); // null-terminate argv[]

        execvp(args[0], args.data());
        exit(1);
    }
    //------------------------------------------------------

    close(in[1]);
    m_fd = out[0];
    auto flags = fcntl(m_fd, F_GETFL);
    fcntl(m_fd, F_SETFL, flags | O_NONBLOCK);

    m_running = true;
}

From my investigation it skips the launching of the command because m_pid isn't 0.

Well the weird thing is, zenity itself exists within the Flatpak container and can be executed normally from the command line.

Hi! Thanks for taking some time to track a bug.

Are you familiar with the fork() system call? It’s used a few lines above the section you refer to, and what it does is create an exact copy of the current process. One of the copies will then have m_pid != 0, that is the process you are debugging, so it’s behaving as expected. The other copy will have m_pid == 0, that is the child process, which in turn will call execvp() and get replaced with zenity.

I do not know which debugger you are using, but if it’s GDB, you may use set follow-fork-mode child in order to debug the child process instead of the parent process after a call to fork().

You're right everything works fine, so I checked on another side, the program log.
Here's the debug log after I had turned on the logging.

which: no matedialog in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)
which: no qarma in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)
which: no kdialog in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)
Gtk-Message: 12:58:23.940: Failed to load module "canberra-gtk-module"
Gtk-Message: 12:58:24.045: Failed to load module "canberra-gtk-module"
Gtk-Message: 12:58:24.045: Failed to load module "canberra-gtk-module"

(zenity:1098): Gtk-WARNING **: 12:58:25.161: Can't open portal file chooser: GDBus.Error:org.freedesktop.portal.Error.InvalidArgument: invalid filter: invalid glob pattern

This invalid glob pattern error seems to indicate the problem. Can you call pfd::settings::verbose(true); somewhere in your code and run it again? This will give more information in the logs.

This is the full log from the start of the program:

sh-5.1$ ./WickedEngineEditor
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

File not found: /tmpWickedVkPipelineCache.data
Created GraphicsDevice_Vulkan (336 ms)
which: no matedialog in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)
which: no qarma in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)
which: no kdialog in (/app/bin:/usr/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/node/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/cargo/bin:/home/megumumpkin/.var/app/com.vscodium.codium/data/python/bin)

[wi::initializer] Initializing Wicked Engine, please wait...
Version: 0.60.26

wi::jobsystem Initialized with [8 cores] [7 threads] (0 ms)

wi::input Initialized (0 ms)
wi::font Initialized (2 ms)
wi::lua Initialized (2 ms)
wi::physics Initialized [Bullet] (0 ms)
wi::texturehelper Initialized (42 ms)
wi::GPUBVH Initialized (66 ms)
wi::gpusortlib Initialized (71 ms)
wi::Ocean Initialized (90 ms)
wi::audio Initialized [FAudio] (71 ms)
wi::image Initialized (106 ms)
wi::HairParticleSystem Initialized (173 ms)
[Shader check] Started checking 84 registered shaders for changes...
[Shader check] All up to date
wi::EmittedParticleSystem Initialized (1197 ms)
wi::renderer Initialized (1757 ms)

[wi::initializer] Wicked Engine Initialized (1851 ms)
File not found: startup.lua
wi::gui Initialized (0 ms)
pfd: zenity --file-selection --filename=/mnt/Data/Projects/Software/WickedEngine/build/Editor --title Open file --separator=
 --file-filter Model formats (.wiscene, .obj, .gltf, .glb)|*.wiscene *.obj *.gltf *.glb  --confirm-overwrite
Gtk-Message: 21:46:59.043: Failed to load module "canberra-gtk-module"
Gtk-Message: 21:46:59.464: Failed to load module "canberra-gtk-module"
Gtk-Message: 21:46:59.464: Failed to load module "canberra-gtk-module"

(zenity:832): Gtk-WARNING **: 21:47:00.637: Can't open portal file chooser: GDBus.Error:org.freedesktop.portal.Error.InvalidArgument: invalid filter: invalid glob pattern

So, this looks like it is zenity failing, not the PFD library. But I’ve been unable to reproduce the problem with some other random flatpaks. Here are a few more questions:

  • what is the version of zenity inside the flatpak?
  • do you have xdg-desktop-portal-gtk installed?
  • What is the version of zenity inside the flatpak?
    3.32.0

  • Do you have xdg-desktop-portal-gtk installed?
    Yes

I tried again the generated command, it errors when it has to include the --file-filter string, i'll try to find the solution on how the command will accept the file formats.

So i did my own modification which is like this (for zenity only), tried and tested on different programs all running within a flatpak environment.

    else if (is_zenity())
    {
        command.push_back("--file-selection");
        command.push_back("--filename=" + default_path);
        command.push_back("--title");
        command.push_back(title);
        command.push_back("--separator=\n");

        for (size_t i = 0; i < filters.size() / 2; ++i)
        {
            command.push_back("--file-filter=" + filters[2 * i] + " | " + filters[2 * i + 1] + "\'");
        }

        if (in_type == type::save)
            command.push_back("--save");
        if (in_type == type::folder)
            command.push_back("--directory");
        if (!(options & opt::force_overwrite))
            command.push_back("--confirm-overwrite");
        if (options & opt::multiselect)
            command.push_back("--multiple");
    }

Can I create a PR for this?

This does not seem correct to me. Quoting is required when you run commands from a shell, but in this case the library is using execvp which does not involve a shell. Adding quotes will actually cause problems for other users.

If adding these "'" characters solves the issue, it indicates that there is a problem with the flatpak environment, maybe the zenity wrapper, or something else.

Is there a way I could try to reproduce the problem here? Is your code available?

To reproduce, install vscodium from flatpak using flatpak install com.vscodium.codium. Download the binary here and extract the zip and open the folder using the vscodium flatpak. Execute the editor program (WickedEngineEditor) using the terminal inside visual studio code, run the program with working directory is in the program folder. To test the dialog, click the blue Load Model button on the top right of the window.

The source repository is here

Zenity only exists within the vscodium environment. Other gtk programs don't even have one.

Anyways, I've got the solution to the problem. On the engine project that I'm working, the way it handles Linux's file dialog has this one small flaw, it has a space after the glob listing string.

zenity --file-selection --filename=/mnt/Data/Projects/Software/WickedEngine/build/Editor --title Open file --separator=
 --file-filter Model formats (.wiscene, .obj, .gltf, .glb)|*.wiscene *.obj *.gltf *.glb  --confirm-overwrite

There's an additional space before --confirm overwrite string, which on the execvp that means that glob segment has additional, illegal space, I've changed the glob part on the project side, rebuilt, and rerun it. Now the file dialog works, no changes to the library.