Training own models

[Schenng]

Hi @taizan.

Thank you for the great project.

I have successfully run PaintsChainer with your supplied models, but cannot train my own models. I am running into the same "green" issue described in #84. (I am starting a new issue because #84 is over a year old).

I have tried many things to solve it such as:

• cuDNN versions
• CUDA versions
• Switching the OpenCV RGB vs BGR formats
• Re-installing everything

The following are the output images after training a very small dataset of 30 images from Pixiv. The more epochs, the more green the picture becomes.

20 Epochs	40 Epochs	100 Epochs

I also had to change the code in train_128.py on like 171.

t_out[i, 0, :] = xp.asarray(batch[i][1]) -> t_out[i, :] = xp.asarray(batch[i][1])

because of the following error:

  File "train_128.py", line 171, in update_core
    t_out[i, 0,:] = xp.asarray(batch[i][1])
  File "cupy/core/core.pyx", line 1553, in cupy.core.core.ndarray.__setitem__
  File "cupy/core/core.pyx", line 3207, in cupy.core.core._scatter_op
  File "cupy/core/elementwise.pxi", line 820, in cupy.core.core.ufunc.__call__
  File "cupy/core/elementwise.pxi", line 404, in cupy.core.core._get_out_args
ValueError: Out shape is mismatched

Setup 1 (Personal machine):
Ubuntu: 16.04
Python: 3.5
GPU: NVIDIA Titan V
CUDA: 9.1
cuDNN: Tried 7.3.1 & 7.3.1
Chainer: 4.4.0

Setup 2 (Google Deep Learning VM):
Debian: 9.5
Python: 3.5.3
GPU: NVIDIA Tesla K80
CUDA: 9.2
cuDNN: 7.2.1
Chainer: 4.5.0

Image array after reading RGB2YUV:
[[[158 82 147]
[158 82 147]
[158 82 147]
...,
[254 128 129]
[255 128 128]
[255 128 128]]

[[159 82 147]
[159 82 147]
[159 82 147]
...,
[248 124 130]
[250 124 130]
[252 124 130]]

[[159 82 147]
[159 82 147]
[159 82 147]
...,
[240 122 130]
[243 122 130]
[246 122 130]]

...,
[[176 97 135]
[199 98 135]
[211 100 136]
...,
[157 82 147]
[156 82 147]
[149 82 147]]

[[164 85 145]
[164 85 145]
[166 86 144]
...,
[163 82 147]
[167 82 147]
[165 82 147]]

[[160 79 152]
[163 79 152]
[160 78 151]
...,
[185 82 147]
[196 82 147]
[199 82 147]]]

Any advice would be highly appreciated.
Thoughts @lllyasviel?

Thanks.
Simon

[taizan]

Hi, can you reconstruct original image using only 1 data without adversarial loss?
If it could not possible , there is some problem on environment.

[lllyasviel]

If you are researcher, and you want to reproduce paintschainer for comparison, I highly recommend to directly use paintschainer web service as your baseline.

If you are currently learning deep learning and want to colorize anime sketches for fun, I think pix2pix is a better start:

https://github.com/pfnet-research/chainer-pix2pix
https://github.com/affinelayer/pix2pix-tensorflow

In fact you can good results very similar to paintschainer V1 from pix2pix, if you have a large enough datasets, after weeks of training. Pix2pix codes are much better prepared and much more easy to read.

[Schenng]

Hi @taizan
Sorry for the delay. I did not have success reconstructing the image using only 1 data without the adversarial loss. I guess it's an environment problem...

@lllyasviel
I have started exploring pix2pix and it seems like a good alternative. Thank you for the advice!