the color of generated pic is not consistent with orginal

[llf1234]

thanks a lot，recently i train a model with your provided code and data，i find the color of generated eyes is not always consistent with the original input（test on other pic which is not in columbia dataset），when i cancel all instance normalization layer in generator， the color of generated eyes become approximate consitent with the original input？ did you pay attention on this phenomenon.

[HzDmS]

I don't really understand what do you mean by 'cancel the instance normalisation'. And I also don't know how many samples have you tested on, and what kind of metric did you use to measure the color shift. But generally speaking, the color shift problem is sort of a common problem for CycleGAN-based approaches (see Figure. 9 in the original CycleGAN paper). It can be alleviated by adding a L_{identity} loss. And also, I wouldn't expect the model could generalise that well on wild images, as ColumbiaGaze only contains 56 subjects. If a dataset with more subjects is publicly available, I believe the model could get an even better performance.

[llf1234]

hi i ask is how this work works in the whole face，for a face，we first crop the right or left eye as paper proposed，and then correct the gaze using your work，last we place the generated eye in the face。 but if the generator has instance normalization layer after conv2，the generated eye has color shift problem，when i use the generated eye to replace the original eye in face pic，it will make the whole face very unnaturual。 if the generator does not have instance normalization layer，the color shift peoblem is gone，but the generated eye tends to smooth ，did you test your model in any whole face picture，even in columin gaze dataset？i have no idea to solve this problem。

…

---Original--- From: "HzDmS"<notifications@github.com> Date: Mon, Aug 12, 2019 23:00 PM To: "HzDmS/gaze_redirection"<gaze_redirection@noreply.github.com>; Cc: "llf1234"<1021789809@qq.com>;"Author"<author@noreply.github.com>; Subject: Re: [HzDmS/gaze_redirection] the color of generated pic is not consistent with orginal (#3) I don't really understand what do you mean by 'cancel the instance normalisation'. And I also don't know how many samples have you tested on, and what kind of metric did you use to measure the color shift. But generally speaking, the color shift problem is sort of a common problem for CycleGAN-based approaches (see Figure. 9 in the original CycleGAN paper). It can be alleviated by adding a L_{identity} loss. And also, I wouldn't expect the model could generalise that well on wild images, as ColumbiaGaze only contains 56 subjects. If a dataset with more subjects is publicly available, I believe the model could get an even better performance. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[HzDmS]

You could probably find this work https://arxiv.org/pdf/1712.03999.pdf helpful.

[llf1234]

sorry about that，why instance normalization layer makes color shift problem？

…

---Original--- From: "HzDmS"<notifications@github.com> Date: Tue, Aug 13, 2019 22:51 PM To: "HzDmS/gaze_redirection"<gaze_redirection@noreply.github.com>; Cc: "llf1234"<1021789809@qq.com>;"Author"<author@noreply.github.com>; Subject: Re: [HzDmS/gaze_redirection] the color of generated pic is not consistent with orginal (#3) You could probably find this work https://arxiv.org/pdf/1712.03999.pdf helpful. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[r06631001]

Hi @llf1234,
I also face the inconsistent color problem.
Did you remove all the instance_norm in the generator to solve the inconsistent color problem?
Or only remove the instance_norm with the scope "in_conv_%d"
Thanks!

[llf1234]

hi，when i remove all layer normalization， the color is consistant with the original image，but it makes the output being smooth

…

---Original--- From: "r06631001"<notifications@github.com&gt; Date: Mon, Mar 9, 2020 14:03 PM To: "HzDmS/gaze_redirection"<gaze_redirection@noreply.github.com&gt;; Cc: "Mention"<mention@noreply.github.com&gt;;"llf1234"<1021789809@qq.com&gt;; Subject: Re: [HzDmS/gaze_redirection] the color of generated pic is not consistent with orginal (#3) Hi @llf1234, I also face the inconsistent color problem. Did you remove all the instance_norm in the generator to solve the inconsistent color problem? Or only remove the instance_norm with the scope "in_conv_%d" Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[zhangqianhui]

How to solve the inconsistent color problem, you can read this paper with coarse-to-fine learning

https://arxiv.org/abs/2004.03064