Testing on DTU scan1 & 4
karimcossentini opened this issue · 6 comments
I tested MSVformer on DTU scan 1 and 4 using weights provided on one drive, however it seems that the ply file generated as output is empty (does not contains 3D points)
filtering: 100% 49/49 [00:00<00:00, 59.34it/s]
Convert mvsnet output to gipuma input
convert depth maps: 0% 0/49 [00:00<?, ?it/s]1769472
convert depth maps: 2% 1/49 [00:00<00:16, 2.93it/s]1769472
convert depth maps: 4% 2/49 [00:00<00:17, 2.66it/s]1769472
convert depth maps: 6% 3/49 [00:01<00:17, 2.60it/s]1769472
convert depth maps: 8% 4/49 [00:01<00:17, 2.58it/s]1769472
convert depth maps: 10% 5/49 [00:01<00:17, 2.49it/s]1769472
convert depth maps: 12% 6/49 [00:02<00:17, 2.48it/s]1769472
convert depth maps: 14% 7/49 [00:02<00:17, 2.44it/s]1769472
convert depth maps: 16% 8/49 [00:03<00:17, 2.37it/s]1769472
convert depth maps: 18% 9/49 [00:03<00:16, 2.40it/s]1769472
convert depth maps: 20% 10/49 [00:04<00:17, 2.18it/s]1769472
convert depth maps: 22% 11/49 [00:04<00:17, 2.16it/s]1769472
convert depth maps: 24% 12/49 [00:05<00:16, 2.21it/s]1769472
convert depth maps: 27% 13/49 [00:05<00:15, 2.29it/s]1769472
convert depth maps: 29% 14/49 [00:05<00:14, 2.37it/s]1769472
convert depth maps: 31% 15/49 [00:06<00:14, 2.30it/s]1769472
convert depth maps: 33% 16/49 [00:06<00:14, 2.32it/s]1769472
convert depth maps: 35% 17/49 [00:07<00:13, 2.33it/s]1769472
convert depth maps: 37% 18/49 [00:07<00:12, 2.48it/s]1769472
convert depth maps: 39% 19/49 [00:07<00:11, 2.62it/s]1769472
convert depth maps: 41% 20/49 [00:08<00:10, 2.74it/s]1769472
convert depth maps: 43% 21/49 [00:08<00:09, 2.87it/s]1769472
convert depth maps: 45% 22/49 [00:08<00:09, 2.96it/s]1769472
convert depth maps: 47% 23/49 [00:09<00:08, 3.02it/s]1769472
convert depth maps: 49% 24/49 [00:09<00:08, 3.03it/s]1769472
convert depth maps: 51% 25/49 [00:09<00:07, 3.13it/s]1769472
convert depth maps: 53% 26/49 [00:10<00:07, 3.10it/s]1769472
convert depth maps: 55% 27/49 [00:10<00:07, 3.10it/s]1769472
convert depth maps: 57% 28/49 [00:10<00:06, 3.14it/s]1769472
convert depth maps: 59% 29/49 [00:11<00:06, 3.18it/s]1769472
convert depth maps: 61% 30/49 [00:11<00:06, 3.10it/s]1769472
convert depth maps: 63% 31/49 [00:11<00:05, 3.14it/s]1769472
convert depth maps: 65% 32/49 [00:11<00:05, 3.20it/s]1769472
convert depth maps: 67% 33/49 [00:12<00:05, 3.11it/s]1769472
convert depth maps: 69% 34/49 [00:12<00:04, 3.14it/s]1769472
convert depth maps: 71% 35/49 [00:12<00:04, 3.14it/s]1769472
convert depth maps: 73% 36/49 [00:13<00:04, 3.11it/s]1769472
convert depth maps: 76% 37/49 [00:13<00:04, 2.64it/s]1769472
convert depth maps: 78% 38/49 [00:14<00:03, 2.76it/s]1769472
convert depth maps: 80% 39/49 [00:14<00:03, 2.89it/s]1769472
convert depth maps: 82% 40/49 [00:14<00:02, 3.02it/s]1769472
convert depth maps: 84% 41/49 [00:15<00:02, 3.13it/s]1769472
convert depth maps: 86% 42/49 [00:15<00:02, 3.07it/s]1769472
convert depth maps: 88% 43/49 [00:15<00:01, 3.11it/s]1769472
convert depth maps: 90% 44/49 [00:15<00:01, 3.18it/s]1769472
convert depth maps: 92% 45/49 [00:16<00:01, 3.15it/s]1769472
convert depth maps: 94% 46/49 [00:16<00:00, 3.16it/s]1769472
convert depth maps: 96% 47/49 [00:16<00:00, 3.17it/s]1769472
convert depth maps: 98% 48/49 [00:17<00:00, 3.17it/s]1769472
convert depth maps: 100% 49/49 [00:18<00:00, 2.62it/s]
Run depth map fusion & filter
./fusibile/fusibile -input_folder /content/MVSFormer/output/scan1/points_mvsnet/ -p_folder /content/MVSFormer/output/scan1/points_mvsnet/cams/ -images_folder /content/MVSFormer/output/scan1/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing
Command-line parameter error: unknown option -input_folder
input folder is /content/MVSFormer/output/scan1/points_mvsnet/
image folder is /content/MVSFormer/output/scan1/points_mvsnet/images/
p folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/
pmvs folder is
numImages is 49
img_filenames is 49
Device memory used: 2312.110107MB
Device memory used: 2312.110107MB
P folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/
numCameras is 49
Camera size is 49
Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees
Selected views: 49
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
Reading normals and depth from disk
Size consideredIds is 49
Reading normal 0
Reading disp 0
Reading normal 1
Reading disp 1
Reading normal 2
Reading disp 2
Reading normal 3
Reading disp 3
Reading normal 4
Reading disp 4
Reading normal 5
Reading disp 5
Reading normal 6
Reading disp 6
Reading normal 7
Reading disp 7
Reading normal 8
Reading disp 8
Reading normal 9
Reading disp 9
Reading normal 10
Reading disp 10
Reading normal 11
Reading disp 11
Reading normal 12
Reading disp 12
Reading normal 13
Reading disp 13
Reading normal 14
Reading disp 14
Reading normal 15
Reading disp 15
Reading normal 16
Reading disp 16
Reading normal 17
Reading disp 17
Reading normal 18
Reading disp 18
Reading normal 19
Reading disp 19
Reading normal 20
Reading disp 20
Reading normal 21
Reading disp 21
Reading normal 22
Reading disp 22
Reading normal 23
Reading disp 23
Reading normal 24
Reading disp 24
Reading normal 25
Reading disp 25
Reading normal 26
Reading disp 26
Reading normal 27
Reading disp 27
Reading normal 28
Reading disp 28
Reading normal 29
Reading disp 29
Reading normal 30
Reading disp 30
Reading normal 31
Reading disp 31
Reading normal 32
Reading disp 32
Reading normal 33
Reading disp 33
Reading normal 34
Reading disp 34
Reading normal 35
Reading disp 35
Reading normal 36
Reading disp 36
Reading normal 37
Reading disp 37
Reading normal 38
Reading disp 38
Reading normal 39
Reading disp 39
Reading normal 40
Reading disp 40
Reading normal 41
Reading disp 41
Reading normal 42
Reading disp 42
Reading normal 43
Reading disp 43
Reading normal 44
Reading disp 44
Reading normal 45
Reading disp 45
Reading normal 46
Reading disp 46
Reading normal 47
Reading disp 47
Reading normal 48
Reading disp 48
Resizing globalstate to 49
Run cuda
Run gipuma
Grid size initrand is grid: 48-36 block: 32-32
Device memory used: 5189.402832MB
Number of iterations is 8
Blocksize is 15x15
Disparity threshold is 0.100000
Normal threshold is 6.283185
Number of consistent points is 2
Cam scale is 1.000000
Fusing points
Processing camera 0
Found 0.00 million points
Processing camera 1
Found 0.00 million points
Processing camera 2
Found 0.00 million points
Processing camera 3
Found 0.00 million points
Processing camera 4
Found 0.00 million points
Processing camera 5
Found 0.00 million points
Processing camera 6
Found 0.00 million points
Processing camera 7
Found 0.00 million points
Processing camera 8
Found 0.00 million points
Processing camera 9
Found 0.00 million points
Processing camera 10
Found 0.00 million points
Processing camera 11
Found 0.00 million points
Processing camera 12
Found 0.00 million points
Processing camera 13
Found 0.00 million points
Processing camera 14
Found 0.00 million points
Processing camera 15
Found 0.00 million points
Processing camera 16
Found 0.00 million points
Processing camera 17
Found 0.00 million points
Processing camera 18
Found 0.00 million points
Processing camera 19
Found 0.00 million points
Processing camera 20
Found 0.00 million points
Processing camera 21
Found 0.00 million points
Processing camera 22
Found 0.00 million points
Processing camera 23
Found 0.00 million points
Processing camera 24
Found 0.00 million points
Processing camera 25
Found 0.00 million points
Processing camera 26
Found 0.00 million points
Processing camera 27
Found 0.00 million points
Processing camera 28
Found 0.00 million points
Processing camera 29
Found 0.00 million points
Processing camera 30
Found 0.00 million points
Processing camera 31
Found 0.00 million points
Processing camera 32
Found 0.00 million points
Processing camera 33
Found 0.00 million points
Processing camera 34
Found 0.00 million points
Processing camera 35
Found 0.00 million points
Processing camera 36
Found 0.00 million points
Processing camera 37
Found 0.00 million points
Processing camera 38
Found 0.00 million points
Processing camera 39
Found 0.00 million points
Processing camera 40
Found 0.00 million points
Processing camera 41
Found 0.00 million points
Processing camera 42
Found 0.00 million points
Processing camera 43
Found 0.00 million points
Processing camera 44
Found 0.00 million points
Processing camera 45
Found 0.00 million points
Processing camera 46
Found 0.00 million points
Processing camera 47
Found 0.00 million points
Processing camera 48
Found 0.00 million points
ELAPSED 1.522794 seconds
Error: no kernel image is available for execution on the device
Writing ply file /content/MVSFormer/output/scan1/points_mvsnet//consistencyCheck-20230529-221442//final3d_model.ply
store 3D points to ply file
I suggest checking whether the depth is correct. If all predicted depth maps are correct, the problem should be caused by the compiling of gipuma. See README for more details.
Tips: You should revise CUDA_NVCC_FLAGS in CMakeLists.txt according the gpu device you used. We set -gencode arch=compute_70,code=sm_70 instead of -gencode arch=compute_60,code=sm_60 with V100 GPUs. For other GPU types, you can follow... (Please refer to the README)
In fact, I am using ground truth depth maps from Depths_raw zip file, so they should be correct, but In case I am using predicted ones how could I check if it's correct ?
and as for the CUDA_NVCC_FLAGS in CMakeLists.txt I am using tesla T4 so I set -gencode arch=compute_75,code=sm_75 and still generating an empty ply file, I am investing quite an effort and time on it to make it up and running but have not figured out a workaround for it yet.
Any suggestions ? thanks
The logs are down below:
model mvsnet <class 'str'>
device None <class 'NoneType'>
config None <class 'NoneType'>
dataset dtu <class 'str'>
testpath /content/MVSFormer/dtu <class 'str'>
testpath_single_scene None <class 'NoneType'>
testlist /content/MVSFormer/lists/dtu/test.txt <class 'str'>
exp_name None <class 'NoneType'>
batch_size 1 <class 'int'>
numdepth 192 <class 'int'>
resume /content/MVSFormer/pretrained_weights/mvs_best.pth <class 'str'>
outdir /content/MVSFormer/output <class 'str'>
display False <class 'bool'>
share_cr False <class 'bool'>
ndepths None <class 'NoneType'>
depth_interals_ratio None <class 'NoneType'>
cr_base_chs 8,8,8 <class 'str'>
grad_method detach <class 'str'>
no_refinement False <class 'bool'>
full_res False <class 'bool'>
interval_scale 1.06 <class 'float'>
num_view 5 <class 'int'>
max_h 1152 <class 'int'>
max_w 1536 <class 'int'>
fix_res False <class 'bool'>
depth_scale 1.0 <class 'float'>
temperature 0.01 <class 'float'>
num_worker 4 <class 'int'>
save_freq 20 <class 'int'>
filter_method gipuma <class 'str'>
conf 0.1,0.1,0.1,0.1 <class 'str'>
thres_view 3 <class 'int'>
thres_disp 1.0 <class 'float'>
downsample None <class 'NoneType'>
dist_base 4.0 <class 'float'>
rel_diff_base 1300.0 <class 'float'>
fusibile_exe_path ./fusibile/fusibile <class 'str'>
prob_threshold 0.5,0.5,0.5,0.5 <class 'str'>
disp_threshold 0.1 <class 'float'>
num_consistent 2.0 <class 'float'>
use_short_range False <class 'bool'>
combine_conf True <class 'bool'>
tmp 1.0 <class 'float'>
tmps 5.0,5.0,5.0,1.0 <class 'str'>
save_all_confs False <class 'bool'>
########################################################################
*Interval_Scale
1.06
dataset test metas: 49 interval_scale:{'scan4': 1.06}
/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py:478: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
drop_path_rate: --- 0.2
!!!No weight in ./pretrained_weights/alt_gvt_small.pth testing should neglect this.
Loading checkpoint: /content/MVSFormer/pretrained_weights/mvs_best.pth ...
/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:3487: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Iter 0/49, Time:3.415274143218994 Res:(1152, 1536)
Iter 1/49, Time:1.397202491760254 Res:(1152, 1536)
Iter 2/49, Time:1.409536600112915 Res:(1152, 1536)
Iter 3/49, Time:1.4168994426727295 Res:(1152, 1536)
Iter 4/49, Time:1.4076523780822754 Res:(1152, 1536)
Iter 5/49, Time:1.4444632530212402 Res:(1152, 1536)
Iter 6/49, Time:1.402644157409668 Res:(1152, 1536)
Iter 7/49, Time:1.5373404026031494 Res:(1152, 1536)
Iter 8/49, Time:1.377190113067627 Res:(1152, 1536)
Iter 9/49, Time:1.3741869926452637 Res:(1152, 1536)
Iter 10/49, Time:1.3821210861206055 Res:(1152, 1536)
Iter 11/49, Time:1.4419941902160645 Res:(1152, 1536)
Iter 12/49, Time:1.4461524486541748 Res:(1152, 1536)
Iter 13/49, Time:1.4321582317352295 Res:(1152, 1536)
Iter 14/49, Time:1.398536205291748 Res:(1152, 1536)
Iter 15/49, Time:1.3773183822631836 Res:(1152, 1536)
Iter 16/49, Time:1.3878734111785889 Res:(1152, 1536)
Iter 17/49, Time:1.3845915794372559 Res:(1152, 1536)
Iter 18/49, Time:1.3941154479980469 Res:(1152, 1536)
Iter 19/49, Time:1.3992080688476562 Res:(1152, 1536)
Iter 20/49, Time:1.4185020923614502 Res:(1152, 1536)
Iter 21/49, Time:1.4437236785888672 Res:(1152, 1536)
Iter 22/49, Time:1.473573923110962 Res:(1152, 1536)
Iter 23/49, Time:1.4169540405273438 Res:(1152, 1536)
Iter 24/49, Time:1.416062355041504 Res:(1152, 1536)
Iter 25/49, Time:1.4074761867523193 Res:(1152, 1536)
Iter 26/49, Time:1.4010851383209229 Res:(1152, 1536)
Iter 27/49, Time:1.4064266681671143 Res:(1152, 1536)
Iter 28/49, Time:1.39436674118042 Res:(1152, 1536)
Iter 29/49, Time:1.403296947479248 Res:(1152, 1536)
Iter 30/49, Time:1.4358813762664795 Res:(1152, 1536)
Iter 31/49, Time:1.4404237270355225 Res:(1152, 1536)
Iter 32/49, Time:1.4343783855438232 Res:(1152, 1536)
Iter 33/49, Time:1.4601478576660156 Res:(1152, 1536)
Iter 34/49, Time:1.477217197418213 Res:(1152, 1536)
Iter 35/49, Time:1.5847489833831787 Res:(1152, 1536)
Iter 36/49, Time:1.410426139831543 Res:(1152, 1536)
Iter 37/49, Time:1.4033558368682861 Res:(1152, 1536)
Iter 38/49, Time:1.4120643138885498 Res:(1152, 1536)
Iter 39/49, Time:1.4281516075134277 Res:(1152, 1536)
Iter 40/49, Time:1.4707086086273193 Res:(1152, 1536)
Iter 41/49, Time:1.4015827178955078 Res:(1152, 1536)
Iter 42/49, Time:1.3987183570861816 Res:(1152, 1536)
Iter 43/49, Time:1.4093499183654785 Res:(1152, 1536)
Iter 44/49, Time:1.4071369171142578 Res:(1152, 1536)
Iter 45/49, Time:1.399526596069336 Res:(1152, 1536)
Iter 46/49, Time:1.4065728187561035 Res:(1152, 1536)
Iter 47/49, Time:1.4020535945892334 Res:(1152, 1536)
Iter 48/49, Time:1.4119088649749756 Res:(1152, 1536)
average time: 1.4612710329951073
filter depth map with probability map
filtering: 100% 49/49 [00:01<00:00, 45.61it/s]
Convert mvsnet output to gipuma input
convert depth maps: 100% 49/49 [00:16<00:00, 2.91it/s]
Run depth map fusion & filter
./fusibile/fusibile -input_folder /content/MVSFormer/output/scan4/points_mvsnet/ -p_folder /content/MVSFormer/output/scan4/points_mvsnet/cams/ -images_folder /content/MVSFormer/output/scan4/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing
Command-line parameter error: unknown option -input_folder
input folder is /content/MVSFormer/output/scan4/points_mvsnet/
image folder is /content/MVSFormer/output/scan4/points_mvsnet/images/
p folder is /content/MVSFormer/output/scan4/points_mvsnet/cams/
pmvs folder is
numImages is 49
img_filenames is 49
Device memory used: 2079.326172MB
Device memory used: 2079.326172MB
P folder is /content/MVSFormer/output/scan4/points_mvsnet/cams/
numCameras is 49
Camera size is 49
Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees
Selected views: 49
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
Reading normals and depth from disk
Size consideredIds is 49
Reading normal 0
Reading disp 0
Reading normal 1
Reading disp 1
Reading normal 2
Reading disp 2
Reading normal 3
Reading disp 3
Reading normal 4
Reading disp 4
Reading normal 5
Reading disp 5
Reading normal 6
Reading disp 6
Reading normal 7
Reading disp 7
Reading normal 8
Reading disp 8
Reading normal 9
Reading disp 9
Reading normal 10
Reading disp 10
Reading normal 11
Reading disp 11
Reading normal 12
Reading disp 12
Reading normal 13
Reading disp 13
Reading normal 14
Reading disp 14
Reading normal 15
Reading disp 15
Reading normal 16
Reading disp 16
Reading normal 17
Reading disp 17
Reading normal 18
Reading disp 18
Reading normal 19
Reading disp 19
Reading normal 20
Reading disp 20
Reading normal 21
Reading disp 21
Reading normal 22
Reading disp 22
Reading normal 23
Reading disp 23
Reading normal 24
Reading disp 24
Reading normal 25
Reading disp 25
Reading normal 26
Reading disp 26
Reading normal 27
Reading disp 27
Reading normal 28
Reading disp 28
Reading normal 29
Reading disp 29
Reading normal 30
Reading disp 30
Reading normal 31
Reading disp 31
Reading normal 32
Reading disp 32
Reading normal 33
Reading disp 33
Reading normal 34
Reading disp 34
Reading normal 35
Reading disp 35
Reading normal 36
Reading disp 36
Reading normal 37
Reading disp 37
Reading normal 38
Reading disp 38
Reading normal 39
Reading disp 39
Reading normal 40
Reading disp 40
Reading normal 41
Reading disp 41
Reading normal 42
Reading disp 42
Reading normal 43
Reading disp 43
Reading normal 44
Reading disp 44
Reading normal 45
Reading disp 45
Reading normal 46
Reading disp 46
Reading normal 47
Reading disp 47
Reading normal 48
Reading disp 48
Resizing globalstate to 49
Run cuda
Run gipuma
Grid size initrand is grid: 48-36 block: 32-32
Device memory used: 4956.618652MB
Number of iterations is 8
Blocksize is 15x15
Disparity threshold is 0.100000
Normal threshold is 6.283185
Number of consistent points is 2
Cam scale is 1.000000
Fusing points
Processing camera 0
Found 0.00 million points
Processing camera 1
Found 0.00 million points
Processing camera 2
Found 0.00 million points
Processing camera 3
Found 0.00 million points
Processing camera 4
Found 0.00 million points
Processing camera 5
Found 0.00 million points
Processing camera 6
Found 0.00 million points
Processing camera 7
Found 0.00 million points
Processing camera 8
Found 0.00 million points
Processing camera 9
Found 0.00 million points
Processing camera 10
Found 0.00 million points
Processing camera 11
Found 0.00 million points
Processing camera 12
Found 0.00 million points
Processing camera 13
Found 0.00 million points
Processing camera 14
Found 0.00 million points
Processing camera 15
Found 0.00 million points
Processing camera 16
Found 0.00 million points
Processing camera 17
Found 0.00 million points
Processing camera 18
Found 0.00 million points
Processing camera 19
Found 0.00 million points
Processing camera 20
Found 0.00 million points
Processing camera 21
Found 0.00 million points
Processing camera 22
Found 0.00 million points
Processing camera 23
Found 0.00 million points
Processing camera 24
Found 0.00 million points
Processing camera 25
Found 0.00 million points
Processing camera 26
Found 0.00 million points
Processing camera 27
Found 0.00 million points
Processing camera 28
Found 0.00 million points
Processing camera 29
Found 0.00 million points
Processing camera 30
Found 0.00 million points
Processing camera 31
Found 0.00 million points
Processing camera 32
Found 0.00 million points
Processing camera 33
Found 0.00 million points
Processing camera 34
Found 0.00 million points
Processing camera 35
Found 0.00 million points
Processing camera 36
Found 0.00 million points
Processing camera 37
Found 0.00 million points
Processing camera 38
Found 0.00 million points
Processing camera 39
Found 0.00 million points
Processing camera 40
Found 0.00 million points
Processing camera 41
Found 0.00 million points
Processing camera 42
Found 0.00 million points
Processing camera 43
Found 0.00 million points
Processing camera 44
Found 0.00 million points
Processing camera 45
Found 0.00 million points
Processing camera 46
Found 0.00 million points
Processing camera 47
Found 0.00 million points
Processing camera 48
Found 0.00 million points
ELAPSED 1.583567 seconds
Error: no kernel image is available for execution on the device
Writing ply file /content/MVSFormer/output/scan4/points_mvsnet//consistencyCheck-20230530-093109//final3d_model.ply
store 3D points to ply file
This issue should be addressed.
I have the same problem. How can I solve it
The cuda version is inconsistent with the fusibile version
So, we need to re-compile the fusible under the current cuda environment.