saved yml file has entries that are problematic (wrong types) that need to be parsed
Closed this issue · 2 comments
Error message:
Traceback (most recent call last):
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/reacton/core.py", line 151, in wrapper
f(*args, **kwargs)
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/solara/components/file_browser.py", line 163, in on_double_click
on_item(item, True)
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/solara/components/file_browser.py", line 139, in on_item
on_file_open(Path(path))
File "/home/philipp/GIT/alrecon/alrecon/pages/__init__.py", line 451, in load_settings
ar.load_app_settings(str(path))
File "/home/philipp/GIT/alrecon/alrecon/components/alrecon.py", line 288, in load_app_settings
self.settings = yaml.load(file_object, Loader=yaml.SafeLoader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/__init__.py", line 81, in load
return loader.get_single_data()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 51, in get_single_data
return self.construct_document(node)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 60, in construct_document
for dummy in generator:
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
value = self.construct_mapping(node)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 218, in construct_mapping
return super().construct_mapping(node, deep=deep)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 143, in construct_mapping
value = self.construct_object(value_node, deep=deep)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 100, in construct_object
data = constructor(self, node)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/philipp/miniconda3/envs/alrecon/lib/python3.12/site-packages/yaml/constructor.py", line 427, in construct_undefined
raise ConstructorError(None, None,
yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:numpy.core.multiarray.scalar'
in "alrecon/settings/default_test_locally_mod.yml", line 9, column 12
There are element added to the ar object's settings attribute that apparently are not in the original definition:
angle_end: !!python/object/apply:numpy.core.multiarray.scalar
- &id001 !!python/object/apply:numpy.dtype
args:
- f8
- false
- true
state: !!python/tuple
- 3
- <
- null
- null
- null
- -1
- -1
- 0
- !!binary |
AAAAAAAAAAA=
angle_start: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAAAA=
dtype: float32
exp_time: 0.0
proj_end: 4001
proj_start: 0
sino_end: 1020
sino_start: 980
worker: local
Removing them from the input file removes the error. The problem with was in the
alrecon.py line 270,
def load_app_settings(self, filename):
with open(filename, "r") as file_object:
self.settings_file.set(path.basename(filename))
self.settings = yaml.load(file_object, Loader=yaml.SafeLoader)
for key, val in self.settings.items():
exec("self." + key + ".set(val)")
logger.info("Loaded settings file: {0}".format(filename))
where introducing a try statement
in
for key, val in self.settings.items():
exec("self." + key + ".set(val)")
resulting in
# some app settings
for key, value in self.settings.items():
try:
exec("self." + key + ".set(value)")
except Exception as e:
print(e)
print(key, " ", value)
could resolve the issue, only updating existing values.
An additional problem was given by entries such as
angle_start: !!python/object/apply:numpy.core.multiarray.scalar
- *id001
- !!binary |
AAAAAAAAAAA=
that would lead to abortion upon reading of the file.
In YAML, one can encounter advanced constructs like custom Python object serialization (!!python/object/apply:) and binary data (!!binary).
I think that we might want to ignore such entries upon loading the YAML file to avoid executing arbitrary Python code or handling specific data types (in our case entries), we don't need. We need to customize the YAML loader in Python.
--->>>>>
adding a construct such as
import yaml
class SafeLoaderIgnoreUnknown(yaml.SafeLoader):
pass
# Define a function that simply ignores complex tags
def ignore_complex_objects(loader, node):
return None # We're ignoring any complex objects
# Register the function to ignore tags
SafeLoaderIgnoreUnknown.add_constructor(None, ignore_complex_objects)
with
self.settings = yaml.load(file_object, Loader=SafeLoaderIgnoreUnknown)
gives as an output
'alrecon' object has no attribute 'angle_end'
angle_end None
'alrecon' object has no attribute 'angle_start'
angle_start None
'alrecon' object has no attribute 'dtype'
dtype float32
'float' object has no attribute 'set'
exp_time 0.0
'alrecon' object has no attribute 'proj_end'
proj_end 4001
'alrecon' object has no attribute 'proj_start'
proj_start 0
'alrecon' object has no attribute 'sino_end'
sino_end 1020
'alrecon' object has no attribute 'sino_start'
sino_start 980
'str' object has no attribute 'set'
worker local
INFO:alrecon:Loaded settings file: alrecon/settings/default_test_locally_mod2.yml
which is desired and drops all from format unknown or not-expected entries.