CUDA_VISIBLE_DEVICES=0 python code/4_test.py --auto_select False
Closed this issue · 4 comments
Hi Thanks,
You did a great pipeline.
Do I missed something, it shows following.
path_mean: [0.0, 0.0, 0.0]
path_std: [0.0, 0.0, 0.0]
At the code/3_train.py I have the same mistake, however, I repaste the WSI file to the all_wsi folder (which I previously deleted).
However in this step, it still shows me error.
Thanks, looking forward to hear from you.
(tf2) qiang@Qiang:~/Desktop/pcr/deepslide-master$ CUDA_VISIBLE_DEVICES=0 python code/4_test.py --auto_select False
############### CONFIGURATION ###############
all_wsi: all_wsi
val_wsi_per_class: 20
test_wsi_per_class: 30
keep_orig_copy: True
num_workers: 8
patch_size: 224
wsi_train: wsi_train
wsi_val: wsi_val
wsi_test: wsi_test
labels_train: labels_train.csv
labels_val: labels_val.csv
labels_test: labels_test.csv
train_folder: train_folder
patches_eval_train: patches_eval_train
patches_eval_val: patches_eval_val
patches_eval_test: patches_eval_test
num_train_per_class: 80000
type_histopath: True
purple_threshold: 100
purple_scale_size: 15
slide_overlap: 3
gen_val_patches_overlap_factor: 1.5
image_ext: jpg
by_folder: True
color_jitter_brightness: 0.5
color_jitter_contrast: 0.5
color_jitter_saturation: 0.5
color_jitter_hue: 0.2
num_epochs: 20
num_layers: 18
learning_rate: 0.001
batch_size: 16
weight_decay: 0.0001
learning_rate_decay: 0.85
resume_checkpoint: False
save_interval: 1
checkpoints_folder: checkpoints
checkpoint_file: xyz.pt
pretrain: False
log_folder: logs
auto_select: True
preds_train: preds_train
preds_val: preds_val
preds_test: preds_test
inference_train: inference_train
inference_val: inference_val
inference_test: inference_test
vis_train: vis_train
vis_val: vis_val
vis_test: vis_test
device: cuda:0
classes: ['a', 's']
num_classes: 2
train_patches: train_folder/train
val_patches: train_folder/val
path_mean: [0.0, 0.0, 0.0]
path_std: [0.0, 0.0, 0.0]
resume_checkpoint_path: checkpoints/xyz.pt
log_csv: logs/log_3292022_233617.csv
eval_model: checkpoints/xyz.pt
threshold_search: (0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9)
colors: ('red', 'white', 'blue', 'green', 'purple', 'orange', 'black', 'pink', 'yellow')
#####################################################
+++++ Running 4_test.py +++++
----- Finding validation patch predictions -----
model loaded from checkpoints/resnet18_e13_va0.59313.pt
testing on 63680 crops from patches_eval_val/DHMC_0046/DHMC_0046
Traceback (most recent call last):
File "code/4_test.py", line 8, in
get_predictions(patches_eval_folder=config.args.patches_eval_val,
File "/home/qiang/Desktop/pcr/deepslide-master/code/utils_model.py", line 623, in get_predictions
for batch_num, (test_inputs, test_labels) in enumerate(dataloader):
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torchvision/datasets/folder.py", line 180, in getitem
sample = self.transform(sample)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 60, in call
img = t(img)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 221, in forward
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/home/qiang/environments/tf2/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 331, in normalize
raise ValueError('std evaluated to zero after conversion to {}, leading to division by zero.'.format(dtype))
ValueError: std evaluated to zero after conversion to torch.float32, leading to division by zero.
My issue related to this part.
Compute the mean and standard deviation for the given set of WSI for normalization.
path_mean, path_std = compute_stats(folderpath=args.all_wsi,
image_ext=args.image_ext)
@liqiang1102 Can you please provide the directory structure of your data?
Thank you for reply. i guess there may exist some incontinence. i will double check.
project
├── all_wsi
│ ├── a
│ │ ├── DHMC_0006.tif
│ │ ├── DHMC_0007.tif
│ │ ├── DHMC_0009.tif
│ │ ├── DHMC_0010.tif
│ │ ├── DHMC_0011.tif
│ │ ├── DHMC_0014.tif
│ │ ├── DHMC_0015.tif
│ │ ├── DHMC_0016.tif
│ │ ├── DHMC_0081.tif
│ │ ├── DHMC_0082.tif
│ │ └── DHMC_0095.tif
│ └── s
│ ├── DHMC_0001.tif
│ ├── DHMC_0002.tif
│ ├── DHMC_0003.tif
│ ├── DHMC_0004.tif
│ ├── DHMC_0008.tif
│ ├── DHMC_0012.tif
│ ├── DHMC_0013.tif
│ ├── DHMC_0017.tif
│ ├── DHMC_0046.tif
│ ├── DHMC_0047.tif
│ └── DHMC_0074.tif
├── code
│ ├── 1_split.py
│ ├── 2_process_patches.py
│ ├── 3_train.py
│ ├── 4_test.py
│ ├── 5_grid_search.py
│ ├── 6_visualize.py
│ ├── 7_final_test.py
│ ├── compute_stats.py
│ ├── config.py
│ ├── pycache
│ │ ├── compute_stats.cpython-38.pyc
│ │ ├── config.cpython-38.pyc
│ │ ├── utils.cpython-38.pyc
│ │ ├── utils_evaluation.cpython-38.pyc
│ │ ├── utils_model.cpython-38.pyc
│ │ ├── utils_processing.cpython-38.pyc
│ │ └── utils_split.cpython-38.pyc
│ ├── run_all.sh
│ ├── utils_evaluation.py
│ ├── utils_model.py
│ ├── utils_processing.py
│ ├── utils.py
│ ├── utils_split.py
│ └── z_preprocessing
│ ├── 1_rename_files.py
│ ├── 2_svs_to_jpg_tiles.py
│ ├── 3_repiece_jpg_tiles.py
│ ├── 4_anonymize.py
│ ├── compress (copy).sh
│ ├── compress_images.py
│ ├── compress.sh
│ ├── wsi
│ ├── wsi_test
│ │ ├── a
│ │ │ ├── DHMC_0095
│ │ │ │ ├── DHMC_0095_0_0.jpg
...................................
│ └── DHMC_0073_4_3.jpg
├── labels_test.csv
├── labels_train.csv
├── labels_val.csv
├── LICENSE
├── MetaData_Release_1.0.csv
├── README.md
├── RELEASES.md
├── wsi_test
│ ├── a
│ │ └── DHMC_0095.tif
│ └── s
│ └── DHMC_0074.tif
├── wsi_train
│ ├── a
│ │ ├── DHMC_0015.tif
│ │ └── DHMC_0016.tif
│ └── s
│ ├── DHMC_0012.tif
│ ├── DHMC_0013.tif
│ └── DHMC_0017.tif
└── wsi_val
├── a
│ ├── DHMC_0081.tif
│ └── DHMC_0082.tif
└── s
├── DHMC_0046.tif
└── DHMC_0047.tif
Stale issue message