Bug fix in masked_loading ROI table reading in case that obs label ID is string of a float
nrepina opened this issue · 0 comments
nrepina commented
fractal_tasks_core/masked_loading.py line 112 assumes that the label ID is stored as a string of an integer (e.g. '1') in ROI table obs, however this might not be the case. The Cellpose task output indeed stores label ID in this format, but the RDCnet task stores as a string of a float (i.e. '1.0'). Thus table reading of an RDCnet output table throws the error below (see traceback below).
Tldr; int('1') = 1, whereas int('1.0') = ValueError: invalid literal for int() with base 10: '1.0'
To generalize, recommended fix is to change lines 112 to:
label_value = int(float(ROI_table.obs[column_name][ROI_positional_index]))
Traceback:
TASK ERROR: Task name: Rdcnet Segmentation 3d Task, position in Workflow: 5
TRACEBACK:
Traceback (most recent call last):
File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 116, in run_single_task
raise e
File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 105, in run_single_task
_call_command_wrapper(
File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 52, in _call_command_wrapper
raise TaskExecutionError(err)
fractal_server.app.runner.exceptions.TaskExecutionError: 2024-07-09 10:29:26,473; INFO; START rdcnet_segmentation_3d task
2024-07-09 10:29:26,474; INFO; zarr_url='/tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0'
2024-07-09 10:29:26,501; INFO; NGFF image has num_levels=5
2024-07-09 10:29:26,501; INFO; NGFF image has coarsening_xy=2
2024-07-09 10:29:26,501; INFO; NGFF image has full-res pixel sizes [0.6, 0.21666666666666667, 0.21666666666666667]
2024-07-09 10:29:26,501; INFO; NGFF image has level-0 pixel sizes (0.6, 0.21666666666666667, 0.21666666666666667)
2024-07-09 10:29:26,511; INFO; data_zyx.shape=(146, 3077, 7730)
2024-07-09 10:29:26,622; INFO; ROI table at /tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/tables/org_ROI_table has attrs: {'encoding-type': 'anndata', 'encoding-version': '0.1.0', 'fractal_table_version': '1', 'instance_key': 'label', 'region': {'path': '../labels/org'}, 'type': 'masking_roi_table'}
2024-07-09 10:29:26,622; INFO; ROI table can be used for masked loading
2024-07-09 10:29:26,622; INFO; use_masks=True
2024-07-09 10:29:26,650; INFO; Helper function `prepare_label_group` returned label_group=<zarr.hierarchy.Group '/labels/nuc'>
2024-07-09 10:29:26,650; INFO; Output label path: /tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/labels/nuc/0
2024-07-09 10:29:26,655; INFO; mask will have shape (146, 3077, 7730) and chunks ((1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), (1000, 1000, 1000, 77), (1000, 1000, 1000, 1000, 1000, 1000, 1000, 730))
2024-07-09 10:29:26,655; INFO; 3D segmentation will be executed
2024-07-09 10:29:26.726435: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-07-09 10:29:28.264298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 30986 MB memory: -> device: 0, name: Tesla V100-SXM2-32GB, pci bus id: 0000:1a:00.0, compute capability: 7.0
2024-07-09 10:29:29,599; INFO; Now starting loop over 12 ROIs
2024-07-09 10:29:29,599; INFO; Now processing ROI 1/12
2024-07-09 10:29:29,784; INFO; [_preprocess_input] image_array.shape=(1, 146, 48, 112)
2024-07-09 10:29:29,784; INFO; [_preprocess_input] region=(slice(0, 146, None), slice(0, 48, None), slice(7119, 7231, None))
2024-07-09 10:29:29,796; INFO; [_preprocess_input] ROI_table_path='/tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/tables/org_ROI_table'
2024-07-09 10:29:29,796; INFO; [_preprocess_input] attrs.asdict()={'encoding-type': 'anndata', 'encoding-version': '0.1.0', 'fractal_table_version': '1', 'instance_key': 'label', 'region': {'path': '../labels/org'}, 'type': 'masking_roi_table'}
Traceback (most recent call last):
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib/python3.9/site-packages/isit/fractal/rdcnet_segmentation_3d.py", line 101, in <module>
run_fractal_task(
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/tasks/_utils.py", line 79, in run_fractal_task
metadata_update = task_function(**pars)
File "pydantic/decorator.py", line 40, in pydantic.decorator.validate_arguments.validate.wrapper_function
File "pydantic/decorator.py", line 134, in pydantic.decorator.ValidatedFunction.call
File "pydantic/decorator.py", line 206, in pydantic.decorator.ValidatedFunction.execute
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib/python3.9/site-packages/isit/fractal/rdcnet_segmentation_3d.py", line 77, in rdcnet_segmentation_3d
return rdcnet_segmentation(
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/isit/fractal/rdcnet_segmentation.py", line 495, in rdcnet_segmentation
new_label_img = seg_fun(**seg_fun_kwarg)
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/isit/fractal/rdcnet_segmentation.py", line 439, in _do_seg3d
new_label_img = masked_loading_wrapper(
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/masked_loading.py", line 238, in masked_loading_wrapper
) = _preprocess_input(image_array, **preprocessing_kwargs)
File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/masked_loading.py", line 112, in _preprocess_input
label_value = int(ROI_table.obs[column_name][ROI_positional_index])
ValueError: invalid literal for int() with base 10: '1.0'