qtsowrap

This is a library that wraps Qt's Linux system integration dependencies in a way so that they are loaded at run time, and thus unneeded during the compile process. The aim is to wrap Qt's entire interface with the operating system.

The source is entirely auto-generated from upstream headers. It is a two-phase process: scripts/collector.py downloads the libraries and extracts the headers, and scripts/gen.py contains the definitions of the libraries and functions to wrap. See the section "Generation" below on how to regenerate the source code after updating these, or just to check.

For now it handles the boilerplate for:

xcb and sublibraries
xkb
fontconfig
freetype

why

Reduce the number of libraries in 'depends' (faster builds). The display and font system libraries no longer need to be built, this library acts as a full replacement from the perspective of the build system.
Optional dependencies (eg Wayland, X11) in otherwise static Qt build. Unlike when directly linking, failing to load libraries is not necessarily fatal and can instead disable the functionality.
- Qt's xcb_xinput could be handled in this way. It's a library that's only present for newer xcb versions, and is used to support XInput2, which is optional (that said, it's likely not needed for us at all). Currently Qt hacks around this by building an internal version of this lib.
Could print more useful errors when a library is missing.
Tighter control over version requirements by restricting which symbols are wrapped.
Optional feature support by being tolerant of (some) missing symbols.
Could potentially do some checks before loading the library (and its dependencies), or log hashes for troubleshooting.

Not all of these are currently implemented.

Building

A CMake build system is used to build the stub library.

mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/.../my/staging/dir
make
make install

Usage

Include the wrapper header instead of the actual header (it will internally include that). The library functions can be called as normally. Just make sure to initialize every library before using it, and check for errors:

#include "xcb-so_wrap.h"

int main()
{
    if (initialize_xcb()) {
        fprintf(stderr, "Unable to load the xcb library. Make sure to install it from your distribution's package repository.\n");
        return 1;
    }
    /* Use xcb functions here. */
    xcb_connect(...);
    return 0;
}

Once a library is bound, it's bound for the lifetime of the process. There is no need to call a deinitialize. Calling initialize multiple times does not make a difference, the dynamic linker will notice that it already loaded the library and re-use it.

Patching Qt

Qt cannot use this library by default. The intent is to keep the changes to a minimum, but it needs to be patched to use the wrapped headers, as well as link against the wrapper library.

The current iteration of the patch can be found in bitcoin PR #29923. It is targeted for Qt 5.15.13 LTS.

A brief overview of the patch for reviewers:

qtbase/src/gui/configure.json: JSON description of qtbase's dependency libraries. Replace the header file names with the wrapped header files, and the -l options with -lqtsowrap -ldl.
qtbase/src/3rdparty/xcb/libxcb: Patch Qt's internally-built libxcb_input to use the wrapped xcb header.
qtbase/src/plugins/platforms/xcb: Qt's xcb platform plugin (libQt5XcbQpa.a / -lxqcb).
- All source files are patched to use the wrapped headers for all xcb libraries except for libxcb_input, which is built as part of 3rdparty.
- Use the wrapped headers for xkbcommon and xkbcommon_x11.
- Initialization is added to QXcbIntegrationPlugin::create. If any of the xcb or xkbcommon libraries cannot be loaded, reject loading the xcb plugin. This is not necessarily fatal, it means that another platform such as wayland could be tried.
qtbase/src/platformsupport/fontdatabases/fontconfig: Qt's fontconfig font database plugn.
- All source files are patched to use the wrapped headers for the fontconfig library.
- Initialization is added to QFontconfigDatabase::QFontconfigDatabase. Load the fontconfig library before any of its methods are used. Failure to load is fatal, this is a required library.
qtbase/src/platformsupport/fontdatabases/freetype: Qt's freetype font engine.
- All source files are patched to use the wrapped headers for the freetype library.
- Initialization is added to qt_getFreetypeData(), which is the earliest point where ft functions might be called. Failure to load is fatal, as this is a required library.

Generation

Generation dependends on pycparser for parsing the input headers:

pip install pycparser

Clean up existing directories (optional):

rm -rf include src_gen include_gen gentmp

To download and verify the original tarballs and extract and generate relevant headers to include.

python3 scripts/collector.py

To re-generate the headers and implementation files in src_gen and include_gen from the headers in include:

python3 scripts/gen.py

This process should be fully deterministic, so to verify that this worked correctly is a matter of re-running the script and checking that there is a clean tree.

Versions

See the table SOURCES in collector.py for the list of library versions used to generate the headers.

Credits

This is based on dynload-wrapper by Hein-Pieter van Braam hp@tmm.cx et al, which is also used by the Godot game engine. This project made the following changes:
- Parse multiple includes consecutively instead of one at a time: this allows for implicit dependencies between headers, e.g. headers that assume some other header to already have been included by the referencing code.
- Handle exported variables. This is necessary for xcb's xcb_randr_id and such.
- Add --include-dir flag to provide additional preprocessor include directory.
- Remove xcb dummy from the fake_libc_include (as we actually want to wrap this library!).

laanwj/qtsowrap