Training model unable to flat horizontal orientation of the image

[AndyXW]

Hello！ I used your Distorted Image code to generate my own training dataset, and I trained the model using this dataset. But it seems that the output model is not able to correct the horizontal orientation of the image, like the picture below shows.

You can see that the left side and the right side of the flated image is also as same as the distort image, keep tilt. But on the vertical direction, like the up side and bottom side have been corrected. The same problem happens on the all the test pictures.
SO I check the output regress of the model, the displacement value of horizontal direction is very small, the max absolute value is only around 0.1. But the displacement value of vertical direction is relatively big, which seems reasonable.
For detecting this problem, I have attempted to check the loss function, but it seems correct. And also I checked the generated label_regress which contains the flow of x and y. I just used the label_regess to flat the distorted image, then the distorted image can be flated in a totally correct way. So the generated training dataset is correct.
Do you have any idea what the problem might be? Thanks for your help in advance.

[AndyXW]

And also I chose one image from training set, and check the output of model of different epoch. Maybe this will give you more information.

[gwxie]

And also I chose one image from training set, and check the output of model of different epoch. Maybe this will give you more information.

Hi,
How many epochs have you trained? Did you start training from scratch?

[AndyXW]

I have trained 300 epochs and the model start training from scratch.

[AndyXW]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

[huxianer]

I have trained 300 epochs and the model start training from scratch.

Yes,I meet the same problem start training from scratch,I think some thing should be fixed, such as network structure, some parameter . Could you give any advice @gwxie @AndyXW

[gwxie]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

[AndyXW]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

[gwxie]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

[AndyXW]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

[gwxie]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

Hello, @AndyXW and @huxianer. Thanks for your feedback! I suggest you try to synthesize more data (about 10k-30k). Or, please simplify the structure of the model and use BN, etc.

[huxianer]

OK! I will try this way and sencerely grateful for your kind help!

[AndyXW]

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

Hello, @AndyXW and @huxianer. Thanks for your feedback! I suggest you try to synthesize more data (about 10k-30k). Or, please simplify the structure of the model and use BN, etc.
Okay, thanks for your help!