All predcition are road
JalongMa opened this issue · 6 comments
Hi, Hengshuang!
This is a valid work for practical application. I've run your code. But something may be wrong. The predictions of network are all road, i.e. every element of 'imPred' is one. I can not locate the problem. I tested many images and got same results. What's wrong with that? thx
@JalongMa Strangely I am also seeing only 1 in imPred (in eval_sub.m). net.forward({input_data}) is giving a 2049x1025x19 tensor which has got max value always in the first dimension, i.e. at [:,:,1]. Same for both pre-trained models. Since I have checked that the image that I have have loaded is valid I am a bit puzzled what is going on. Have you solved the issue?
@revilokeb I only got the right or reasonable predictions under CPU mode. You can try it.
@JalongMa ok, thank you. However, I have reinvestigated the issue and I was having the following problem which resulted in me obtaining only 1s in imPred:
- my hardware setup is a machine with one Pascal and two Maxwell GPUs
- normally I am running Ubuntu 16 and CUDA 8 + cuDNN 5.1, but I have got another setup on that machine with Ubuntu 14, CUDA 7.5 which I used for ICNet (I removed cuDNN 5.1 and replaced with cuDNN 4)
- With that Ubuntu 14 / CUDA 7.5 setup I was running inference on the Pascal GPU (which happened to have GPU ID = 0). But Pascal GPUs require CUDA 8 to work properly.
- So I just changed GPU ID to one of the Maxwell GPUs and it is now working flawlessly on GPU
Maybe you are also running CUDA 7.5 on Pascal?
@revilokeb As you said, I ran this with CUDA 7.5 on Pascal. Maybe there are something different between Pascal and Maxwell. If you got only 1s under GPU mode, that's inconsistent with claims in the paper. Anyway, authors don't provide training scripts. This really bothers me....
@JalongMa As I said, on my Titan X Maxwell the code is running perfectly. I am assuming that they have performed their experiments also on Titan X Maxwell (see their paper https://arxiv.org/pdf/1704.08545.pdf, section 6) and not Pascal. It is known that CUDA 7.5 does not neccessarily run without issues on Pascal, e.g. look at https://devtalk.nvidia.com/default/topic/955165/cuda-setup-and-installation/pascal-titan-x-with-cuda-7-5/ and http://docs.nvidia.com/cuda/pascal-compatibility-guide/index.html#axzz4lCdXo164
@revilokeb OK, thank you. I'll try out on Maxwell TITAN. Good luck !