DCAN-Labs/BIBSnet

Read only file system error

Closed this issue · 9 comments

What happened?

Erroring out because of read only file system:
Traceback (most recent call last):
File "/opt/conda/bin/nnUNet_predict", line 33, in
sys.exit(load_entry_point('nnunet', 'console_scripts', 'nnUNet_predict')())
File "/opt/conda/bin/nnUNet_predict", line 25, in importlib_load_entry_point
return next(matches).load()
File "/opt/conda/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/opt/conda/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 843, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/cabinet/nnUNet/nnunet/inference/predict_simple.py", line 19, in
from nnunet.inference.predict import predict_from_folder
File "/home/cabinet/nnUNet/nnunet/inference/predict.py", line 21, in
from batchgenerators.augmentations.utils import resize_segmentation
File "/opt/conda/lib/python3.8/site-packages/batchgenerators/augmentations/utils.py", line 22, in
from skimage.transform import resize
File "/opt/conda/lib/python3.8/site-packages/skimage/init.py", line 141, in
from .data import data_dir
File "", line 1039, in _handle_fromlist
File "/opt/conda/lib/python3.8/site-packages/lazy_loader/init.py", line 76, in getattr
submod = importlib.import_module(submod_path)
File "/opt/conda/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/opt/conda/lib/python3.8/site-packages/skimage/data/_fetchers.py", line 270, in
_init_pooch()
File "/opt/conda/lib/python3.8/site-packages/skimage/data/_fetchers.py", line 247, in _init_pooch
os.makedirs(data_dir, exist_ok=True)
File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 1 more time]
File "/opt/conda/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/adp9359'

What command did you use?

#!/bin/bash
#SBATCH --account=p31681  ## YOUR ACCOUNT pXXXX or bXXXX
#SBATCH --partition=gengpu
#SBATCH --nodes=1 ## how many computers do you need
#SBATCH --ntasks-per-node=1 ## how many cpus or processors do you need on each computer
#SBATCH --gres=gpu:a100:1  ## type of GPU requested, and number of GPU cards to run on
#SBATCH --time=04:00:00 ## how long does this need to run (remember different partitions have restrictions on this param)
#SBATCH --mem=40G ## how much RAM do you need per CPU (this effects your FairShare score so be careful to not ask for more than you need))
#SBATCH --job-name=run_ibeat ## When you run squeue -u NETID this is how you can identify the job
#SBATCH --output=cabinet_test_1007.log ## standard out and standard error goes to this file

module purge
module load singularity

singularity run --nv --cleanenv --no-home \
-B /projects/b1174/cabinet_test/bidsdir:/input \
-B /projects/b1174/cabinet_test/outdir:/output \
-B /projects/b1174/cabinet_test/param_files/param_file.json:/param_file.json \
/projects/b1174/singularity_images/cabinet:latest.sif \
/input /output participant -jargs /param_file.json -end postbibsnet -v -participant 1007

What version of CABINET are you using?

latest from singularity

Relevant log output

Traceback (most recent call last):
  File "/opt/conda/bin/nnUNet_predict", line 33, in <module>
    sys.exit(load_entry_point('nnunet', 'console_scripts', 'nnUNet_predict')())
  File "/opt/conda/bin/nnUNet_predict", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/opt/conda/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/cabinet/nnUNet/nnunet/inference/predict_simple.py", line 19, in <module>
    from nnunet.inference.predict import predict_from_folder
  File "/home/cabinet/nnUNet/nnunet/inference/predict.py", line 21, in <module>
    from batchgenerators.augmentations.utils import resize_segmentation
  File "/opt/conda/lib/python3.8/site-packages/batchgenerators/augmentations/utils.py", line 22, in <module>
    from skimage.transform import resize
  File "/opt/conda/lib/python3.8/site-packages/skimage/__init__.py", line 141, in <module>
    from .data import data_dir
  File "<frozen importlib._bootstrap>", line 1039, in _handle_fromlist
  File "/opt/conda/lib/python3.8/site-packages/lazy_loader/__init__.py", line 76, in __getattr__
    submod = importlib.import_module(submod_path)
  File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/opt/conda/lib/python3.8/site-packages/skimage/data/_fetchers.py", line 270, in <module>
    _init_pooch()
  File "/opt/conda/lib/python3.8/site-packages/skimage/data/_fetchers.py", line 247, in _init_pooch
    os.makedirs(data_dir, exist_ok=True)
  File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 1 more time]
  File "/opt/conda/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/adp9359'

Add any additional information or context about the problem here.

Thank you!

Hi @ajdneuro - I suggest pulling the tagged 2.4.0 version of cabinet. This was an older bug. If that doesn't fix it, specify a working directory, and that should do the trick.

Thanks trying now!

Thank you for your response, I pulled 2.4.0 and adding a working dir and I am still getting the read only file system error, any thoughts?
Screenshot 2023-05-23 at 4 41 56 PM

Are you giving relative paths, or are those absolute? Also, are you getting the exact same error?

Hi,
These are absolute and yes it is saying my home directory (/home/adp9359) is read only, thank you!:
Screenshot 2023-05-23 at 4 51 55 PM

It looks like you need to make the directory you bind as the working dir writeable.

@ajdneuro - someone on our team said they get this error when using the --no-home flag. Try removing that and see if that fixes it. Thanks!

@ajdneuro - I am going to close off this issue. Please re-open it if the problem persists.