Mysterious error in source.convert when using tomlkit instead of tomlib

[MementoRC]

tomlib is not available in early version of python. I used tomlkit instead to read a supplemental scikit_config to modify the settings prior to calling the BUilder to experiment with buildiing TARGETS and installing associated COMPONENTS. But I encounter a very incomprehensible issue where the dictionary created by tomlkit (in fact a TOMLDocument) create this fail:

xxx/.conda/envs/python-3.11/lib/python3.11/typing.py", line 462, in __call__
E                 |     raise TypeError(f"Cannot instantiate {self!r}")
E                 | TypeError: Cannot instantiate typing.Union
E                 | Field tool.scikit-build.cmake.define
E                 +------------------------------------

It occurs in:

        if raw_target is Union and type(item) in get_args(target):
            return item
        if raw_target is Literal:
            if item not in get_args(_process_union(target)):
                msg = f"{item!r} not in {get_args(_process_union(target))!r}"
                raise TypeError(msg)
            return item
        if callable(raw_target):
            return raw_target(item)  # <-

But it should be routed to if raw_target is Union and type(item) in get_args(target):

In my code, I tried trahsferring the TOMLDocument to a simple dict and assert that both tomlkit and tomlib dictionary are the same, but the tomlib is passing while the tomlkit fails

This is the updating function (butchered from desperation at understanding this issue):

def update_scikit_config_from_plugin(root_dir: str, config: dict[str, Any]) -> dict[str, Any]:
    """
    Reads scikit-build-core TOML configuration from its configuration file
    defined in the plugin configuration 'toml-file' entry.

    Updates scikit-build-core TOML configuration based upon entries of
    the plugin configuration.
    Note: The 'targets' entry of the plugin configuration must NOT be passed
    to the configuration phase of CMake/Builder

    :param root_dir: Root directory of the project
    :param config: Plugin configuration (TOML)
    :return: Updated configuration
    """

    def _get_TOML_scikit_config(path) -> dict[str, Any]:
        toml: TOMLDocument = TOMLFile(path).read().get('tool', {}).setdefault('scikit-build', {})
        return {k: v for k, v in toml.items()}

    def _update_config_defaults(sk_config: dict[str, Any]) -> dict[str, Any]:
        # Retrieve the configuration entries from the plugin configuration
        sdist: dict[str, Any] = sk_config.setdefault('sdist', {})
        wheel: dict[str, Any] = sk_config.setdefault('wheel', {})

        # We disable sdist ('sdist' build should be ignored by the hook initialization)
        if sdist.get('cmake', False):
            log.warning("scikit hatch plugin disables the 'cmake' option in the 'sdist' section")

        sdist['cmake'] = False
        sdist['include'] = []

        # Override the eventual 'packages' in the scikit-build-core config (these should be handled by hatchling)
        if wheel.get('packages', False):
            log.warning('scikit hatch plugin resets the wheel packages to an empty list')

        # wheel['cmake'] = True
        wheel['packages'] = []
        wheel['platlib'] = 'platlib'

        return sk_config

    # Typically, the scikit-build-core configuration is stored in the 'pyproject.toml' file
    tomlfile_for_scikit: str = config.get('toml-file', 'pyproject.toml')

    if not (toml_file_path := Path(os.path.join(root_dir, tomlfile_for_scikit))).exists():
        msg = f'Could not find {tomlfile_for_scikit} in the project root directory'
        raise FileNotFoundError(msg)

    # Read settings from the tool.scikit-build section of the pyproject.toml
    scikit_config: dict[str, Any] = _get_TOML_scikit_config(toml_file_path)

    with toml_file_path.open('rb') as ft:
        import tomllib
        scikit_config0: dict[str, Any] = (tomllib.load(ft)).get('tool', {}).setdefault('scikit-build', {})
    assert scikit_config == scikit_config0

    # Entries are dependent upon scikit-build-core API (current v0.8.2)
    scikit_config0 = _update_config_defaults(scikit_config0)
    scikit_config = _update_config_defaults(scikit_config)
    assert scikit_config == scikit_config0

    # Update the scikit-build-core directories (the default setting works well with hatchling)
    #if (_res := 'source-dir') and (_dir := config.get(_res, None)):
    #    scikit_config.setdefault('cmake', {})[_res] = _dir
    #    scikit_config0.setdefault('cmake', {})[_res] = _dir

    print(f"scikit_config: {((scikit_config))}")
    print(f"scikit_config: {(scikit_config0)}")
    assert scikit_config == scikit_config0
    return {'tool': {'scikit-build': scikit_config}}

[henryiii]

Tomli is the backport of tomllib. Does that work? I can still check to see why tomlkit is failing; but that makes special versions of dicts and things so it can 100% round trip and that might trip us up.

you only need tomlkit when writing a document that you read back out.

[MementoRC]

Great, that works, thanks. Closing the issue