Different behaviour in version 3.6.8

[tsinggggg]

Operating System

1. Name/Distribution: macOS(tried on linux as well, same behaviour)
2. Version: 12.6

Version of Python and packages in use at the time of the issue.

1. Distribution:
2. Python Version: 3.9.12
3. Version of yamlpath installed: 0.17.17
4. Version of ruamel.yaml installed: 3.6.8

Minimum sample of YAML (or compatible) data necessary to trigger the issue

from types import SimpleNamespace

from ruamel.yaml import YAML
from yamlpath.wrappers import ConsolePrinter
from yamlpath import Processor
from yamlpath.enums.yamlvalueformats import YAMLValueFormats


if __name__ == '__main__':
    log = ConsolePrinter(
        SimpleNamespace(quiet=True, verbose=False, debug=False)
    )
    config = {"key1": [1, 2, 3],
              "key2": {
                  "key3": "value1",
                  "key4": "value2",
              }
              }
    processor = Processor(log, config)
    processor.set_value("key2.key3", "asdf", value_format=YAMLValueFormats.DEFAULT)
    print(list(processor.get_nodes("key2.key3"))[0].node)

Complete steps to reproduce the issue when triggered via:

Running the script above

Expected Outcome

printing "asdf"

Actual Outcome

printing "value1"

Screenshot(s), if Available

I am not sure if this is the intended usage of this package. But I have a question about this change, why does it accepts both CommentedSeq, list but not dict?

Sorry if I am not using this package the intended way.

[wwkimball]

The Python dict doesn't support the required insert method for certain changes to YAML (notably whenever the key is an Alias, which is changed). Internally, ruamel.yaml (on which YAMLPath is based) doesn't use dicts, but rather CommentedMaps, which are themselves based on ordereddict; both of which support insert operations.

Allowing bare dicts to be used was a bomb waiting to go off. A Python stack dump was inevitable. A recent update to the Python linters and static code analyzers detected the issue, forcing me to drop off support for bare dicts in the update method. In retrospect, I should have added a new error message to indicate when an impossible operation was attempted rather than simply delete the partial capability.

I'll take this as a bug secondary to not thinking the change all the way through. In the moment, I just wanted the code analysis error to go away since I was focused on an issue that had nothing to do with that annoyance.

[wwkimball]

In the mean-time, you can:

1. Downgrade to the previous version of the yamlpath module.
2. Use ruamel.yaml.comments.CommentedMap rather than dict.

[tsinggggg]

Thanks for the response, yes we ended up fixing ==3.6.7 instead of ~=3.6.x.

It'd be great if this breaking change goes into a new minor release instead of patch, this way we are kind of force to change things in the future in case of any future vulnerabilities.

But anyway, thanks for the explanation!

[wwkimball]

It'd be great if this breaking change goes into a new minor release instead of patch, this way we are kind of force to change things in the future in case of any future vulnerabilities.

It may seem like a "breaking change" to you, but you're using this module in a way it wasn't designed to be used; your data is neither YAML, EYAML, nor JSON. From the perspective of the Python code analyzer that forced this change, I fixed a bug. It so happens that fixing my bug created a bug for you, but only because you're using out-of-spec data.

Because I respect that people often use code in unanticipated ways, I'm going to add a special case in the code to handle your unexpected use-case. You likely aren't the only person trying to use this module against raw Python data structures rather than the expressly supported use-case data types.

[wwkimball]

Full support for bare Python dict and collections.OrderedDict has been added in version 3.6.9, which I released a few minutes ago. New unit tests have been added expressly to assert this support continues to work for all future releases. Please update to the latest version and confirm this addition works in your use-case.