Vladkryvoruchko/PSPNet-Keras-tensorflow

Cityscapes weights

jmtatsch opened this issue · 9 comments

jmtatsch commented

Hi Vlad,

I am trying to convert the original pspnet101_cityscapes weights from https://github.com/hszhao/PSPNet to keras.

python weight_converter.py pspnet101_cityscapes_713.prototxt pspnet101_cityscapes.caffemodel

In PSPNet-Keras-tensorflow I adjusted:

WEIGHTS = 'pspnet101_cityscapes.npy'
DATA_MEAN = np.array([[[72.78044, 83.21195, 73.45286]]]) # RGB
x = Conv2D(19, (1, 1), strides=(1, 1), name="conv6")(x) # changed the classes from 150 to 19

PSPNet runs through without errors, unfortunately, the output for a test image remains indiscernible:

test
out

Do you have an idea what I could be missing?

jmtatsch commented

Changed back the means to the default as apparently pspnet did not use different means when training on different datasets, but still:

test

jefequien commented

The architecture for pspnet101_cityscapes_713.prototxt is different from the one specified in layers_builder.py. To load those weights, you need to convert that model into keras.

jmtatsch commented

@hujh14 thank you so much for figuring this out. It seems to work now :)

test3
blended

I will open a pull request later..

hellochick commented

@hujh14 Hey, Could you tell me the difference between directly load weight and load model to keras first?
Since I want to re-implement PSPNet in pure tf for research, but I got bad result on it.

Thanks!!

jmtatsch commented

@hellochick the original authors used a deeper 101 layer resnet in their pspnet for cityscapes. My mistake was that i tried to convert the weights of a 101 layer network to a 50 layer architecture. Of course the architecture needed a deeper resnet also.

  • ❤️1
hellochick commented

@jmtatsch Thank you for answering, I got it. I try to turn your model into tensorflow, but the result didn't meet my expectation:
test
Do you have any idea about the strange result ( Since most of part are predicted correctly except the trees and buildings ) ?
Or is there any difference between keras and tf operations ?

Thank you a lot !

jmtatsch commented

@hellochick can you also edit in the original image above? It definitely has more trouble with the trees than I would expect. I am also still missing 1.7% to reproduce the published results. Could also be the preprocessing/interpolation that is different in a matlab/caffe pipeline.

hellochick commented

@jmtatsch Hey, here is the origin image:
test
And following image is result produced by your model
test_out_seg
It looks perfect