Super-resolution with Light Field data
add model SR_RGB_Bicubic_VarHisto:
-
still in RGB, 3 channels combined. The upscale layer that brings 48x48 patch to 96x96 is concatenated with the bicubic interpolated input of size 96x96x3;
-
add tf.summary.histogram() to observe the behavior(distributution) of the weights and biases of encoder and decoder. Encoder has only weights and biases for downsampling convolutions. Decoder has weights and biases for upsampling deconvolutions to construct symmetrical autoencoder structure, weights and biases for upscaling deconvolutions to achieve super resolution. The other variables for concatennate and the final 1-by-1 convolution are not included.
add model SR ColorSpace Channel Skip:
-
"config_autoencoder.py":
row 36: added flag for whether to use bicubic interpolated input as skip connection patch in upscaling phase;
row 98-103, 127-132: moved the layout in upscale-layers to distinguish the number of feature maps of certain channel(s);
row 144-188: reduced number of layers such that the network could be run on single GPU machines :), should be changed back when the final structure is determined;
row 195-199: added layout of upscale-layer if no skip connection is used for certain channel(s), mainly because of the number of feature maps;
row 246/259: added flag in decoder-config for whether using skip connection for certain channel(s); -
"layers.py":
row 367: added class "decoder-variables-2D-no-SC", mainly because when no skip connection, the layout is different in number of feature maps and the layout cannot be changed directly by operations in "cnn-autoencoder.py" since it would also change the value in the "self" structure; -
"cnn_autoencoder.py":
row 30: bicubic interpolation flag;
row 225: skip connection flag belonging to each decoder;
row 227-228: generating bicubic interpolated input as a skip connection patch in upscale-phase;
row 259-272: if no skip connection is used, then also skip generating the variables for the patches of skip connections;
row 277-284: generating decoder-2D variables according to whether using skip connection;
row 304-319: with/without skip connection in decoder of certain channel(s);
row 330-335: layout changes due to whether using skip connection;
row 337-340: layout changes due to whether using bicubic interpolated inputs;
row 360-361: with/without bicubic interpolated inputs; -
potential problems:
(a). we are using quite many flags in the model, the layouts of layers are still kind of hard-coded, right now I don't have better way to make it very clean;
(b). since the bottleneck now is merged, if the decoder of color channels doesn't use skip connection, the decoder would have much fewer number of feature maps. As one result of it, the first decoder layer right after the bottle neck would dramatically decrease the number of features from 576(=192 * patch-weight, row 247 in "cnn-autoencoder.py") to 160. With the skip connection it would be from 576 to 480(=160 * patch-weight). row 247 in "cnn-autoencoder.py" cannot be changed according to whether we use skip connection, or it would cause the fracture of the network at bottleneck.
(c). another result of not using skip connection is that, because of fewer number of feature maps, we would have much smaller room to increase the number of layers in upscaling phase.
(d). I ran the model with L having skip conncetion and ab not for a little while, the loss of L channel is still satisfiying...:(