gianthk/alrecon

saved yml file has entries that are problematic (wrong types) that need to be parsed

Closed this issue · 2 comments

Error message:

Traceback (most recent call last):
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/reacton/core.py", line 151, in wrapper
    f(*args, **kwargs)
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/solara/components/file_browser.py", line 163, in on_double_click
    on_item(item, True)
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/solara/components/file_browser.py", line 139, in on_item
    on_file_open(Path(path))
  File "/home/philipp/GIT/alrecon/alrecon/pages/__init__.py", line 451, in load_settings
    ar.load_app_settings(str(path))
  File "/home/philipp/GIT/alrecon/alrecon/components/alrecon.py", line 288, in load_app_settings
    self.settings = yaml.load(file_object, Loader=yaml.SafeLoader)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 427, in construct_undefined
    raise ConstructorError(None, None,
yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:numpy.core.multiarray.scalar'
  in "alrecon/settings/default_test_locally_mod.yml", line 9, column 12

There are element added to the ar object's settings attribute that apparently are not in the original definition:

angle_end: !!python/object/apply:numpy.core.multiarray.scalar
- &id001 !!python/object/apply:numpy.dtype
  args:
  - f8
  - false
  - true
  state: !!python/tuple
  - 3
  - <
  - null
  - null
  - null
  - -1
  - -1
  - 0
- !!binary |
  AAAAAAAAAAA=
angle_start: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
  AAAAAAAAAAA=
dtype: float32
exp_time: 0.0
proj_end: 4001
proj_start: 0
sino_end: 1020
sino_start: 980
worker: local

Removing them from the input file removes the error. The problem with was in the
alrecon.py line 270,

def load_app_settings(self, filename):
    with open(filename, "r") as file_object:
        self.settings_file.set(path.basename(filename))
        self.settings = yaml.load(file_object, Loader=yaml.SafeLoader)


    for key, val in self.settings.items():
        exec("self." + key + ".set(val)")

    logger.info("Loaded settings file: {0}".format(filename))

where introducing a try statement

in

for key, val in self.settings.items():
     exec("self." + key + ".set(val)")

resulting in

        # some app settings
        for key, value in self.settings.items():
            try:
                exec("self." + key + ".set(value)")
            except Exception as e:
                print(e)
                print(key, "    ", value)

could resolve the issue, only updating existing values.

An additional problem was given by entries such as

angle_start: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
  AAAAAAAAAAA=

that would lead to abortion upon reading of the file.

In YAML, one can encounter advanced constructs like custom Python object serialization (!!python/object/apply:) and binary data (!!binary).
I think that we might want to ignore such entries upon loading the YAML file to avoid executing arbitrary Python code or handling specific data types (in our case entries), we don't need. We need to customize the YAML loader in Python.

--->>>>>

adding a construct such as

import yaml

class SafeLoaderIgnoreUnknown(yaml.SafeLoader):
    pass

# Define a function that simply ignores complex tags
def ignore_complex_objects(loader, node):
    return None  # We're ignoring any complex objects

# Register the function to ignore tags
SafeLoaderIgnoreUnknown.add_constructor(None, ignore_complex_objects)

with

self.settings = yaml.load(file_object, Loader=SafeLoaderIgnoreUnknown)

gives as an output

'alrecon' object has no attribute 'angle_end'
angle_end      None
'alrecon' object has no attribute 'angle_start'
angle_start      None
'alrecon' object has no attribute 'dtype'
dtype      float32
'float' object has no attribute 'set'
exp_time      0.0
'alrecon' object has no attribute 'proj_end'
proj_end      4001
'alrecon' object has no attribute 'proj_start'
proj_start      0
'alrecon' object has no attribute 'sino_end'
sino_end      1020
'alrecon' object has no attribute 'sino_start'
sino_start      980
'str' object has no attribute 'set'
worker      local
INFO:alrecon:Loaded settings file: alrecon/settings/default_test_locally_mod2.yml

which is desired and drops all from format unknown or not-expected entries.