how to train the SR model for scale 4
xujinchang opened this issue · 13 comments
I have met some questions when i want to train a model for scale = 4, the following are what i did
Firstly , i ran the image_utils.py , i changed the scaling_factor from 2 to 4 and changed the trup_upscale form False to True, then i got the train images in the floder.
Then ,i ran the three lines in test.py:
if name == "main":
path = r""
val_path = "val_images/"
scale = 4
sr = models.ImageSuperResolutionModel(scale)
sr.create_model()
sr.fit(nb_epochs=250)
however i met the errors :
Using Theano backend.
Training model : ImageSuperResolutionModel
Epoch 1/250
Found 23552 images.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/software/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/software/local/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 425, in data_generator_task
generator_output = next(generator)
File "/local/home/share/xujinchang/download/Image-Super-Resolution-master/img_utils.py", line 216, in image_generator
batch_x[i] = img.transpose((2, 0, 1))
ValueError: could not broadcast input array from shape (3,16,16) into shape (3,64,64)
Traceback (most recent call last):
File "train_sr.py", line 11, in
sr.fit(nb_epochs=250)
File "/local/home/share/xujinchang/download/Image-Super-Resolution-master/models.py", line 487, in fit
return super(ImageSuperResolutionModel, self).fit(batch_size, nb_epochs, save_history, history_fn)
File "/local/home/share/xujinchang/download/Image-Super-Resolution-master/models.py", line 100, in fit
nb_val_samples=val_count)
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 1417, in fit_generator
'or (x, y). Found: ' + str(generator_output))
Exception: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
by the way, i didn't do anything in model.py.
Next I changed the models.py class BaseSuperResolutionModel(object) :
self.type_true_upscaling = True
it happened the different errors like these:
Using Theano backend.
Training model : ImageSuperResolutionModel
Epoch 1/250
Found 23552 images.
Traceback (most recent call last):
File "train_sr.py", line 11, in
sr.fit(nb_epochs=250)
File "/local/home/share/xujinchang/download/Image-Super-Resolution-master/models.py", line 487, in fit
return super(ImageSuperResolutionModel, self).fit(batch_size, nb_epochs, save_history, history_fn)
File "/local/home/share/xujinchang/download/Image-Super-Resolution-master/models.py", line 100, in fit
nb_val_samples=val_count)
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 1443, in fit_generator
class_weight=class_weight)
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 1215, in train_on_batch
check_batch_dim=True)
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 961, in _standardize_user_data
exception_prefix='model input')
File "/home/software/local/lib/python2.7/site-packages/keras/engine/training.py", line 108, in standardize_input_data
str(array.shape))
Exception: Error when checking model input: expected input_1 to have shape (None, 3, 32, 32) but got array with shape (128, 3, 16, 16)
I would appreciate it if you can help me with these problems.
I have tried to train the SR for scale = 2 ,it works well.
Epoch 1/250
Found 23552 images.
14464/23552 [=================>............] - ETA: 102s - loss: 0.0201 - PSNRLoss: 19.0624
As for scale =4, where shoud i modify
@xujinchang Disable the true upscaling flag (set it to False). It is not meant for ImageSuperResolution model.
True Upscaling models are : ESPCNN (Efficient Subpixel Convolution NN)
@titu1994 that is to say ,in the image_utils.py, i shouldn't change the true_upscale form False to True .if i wanted to train the SRCNN for scale = 4, only i should do is to set the scaling_factor from 2 to 4 , and in the models.py , i shouldn't change anything.
Just now ,i tried to train ESPCNN, as for ESPCNN, the train_images should do as above?(the image_utils.py , i changed the scaling_factor from 2 to 4 and changed the true_upscale form False to True) in the models.py,
except for the true upscaling flag (set it to False), what else should i modify?
Thanks a lot
I have to look at the code but that's how I designed it.
For ESPCNN, set scaling factor to 4 and true upscale to true.
For ImageSuperResolution, set scale factor to 4 in images.py.
For both of these, When in the training script, the constructor of all models takes an argument scale. Set that to 4 as well.
@titu1994 but in the models.py , the initial height and width of image shouldn't change?
@titu1994 i ran the code once again as you told, For ImageSuperResolution, set scale factor to 4 in images.py , i got 64*64 image, then i run training script , scale = 4, it runs like that
Exception: Error when checking model input: expected input_1 to have shape (None, 3, 32, 32) but got array with shape (128, 3, 64, 64)
in the models.py class ImageSuperResolutionModel(BaseSuperResolutionModel):
def create_model(self, height=32, width=32, channels=3, load_weights=False, batch_size=128),
i think it should modify, change 32 to 64
except that , the class BaseSuperResolutionModel(object):
def create_model(self, height=32, width=32, channels=3, load_weights=False, batch_size=128)
should also change to 64
Yes change it. It's a bug. It should automatically change the input shape.
Having changed it , i can run the SRCNN for scale = 4 successfully, but when i use the same method for ESPCNN,
class BaseSuperResolutionModel(object):
def create_model(self, height=64, width=64, channels=3, load_weights=False, batch_size=128):
class EfficientSubPixelConvolutionalSR(BaseSuperResolutionModel):
def create_model(self, height=16, width=16, channels=3, load_weights=False, batch_size=128):
it met the error
Exception: Error when checking model target: expected convolution2d_1 to have shape (None, 3, 32, 32) but got array with shape (128, 3, 64, 64)
i found the x = SubPixelUpscaling(r=2, channels=self.n, name='sr_res_upscale1_%d' % id)(x), where r infers to the scale, does it cause the error?
Yes. That needs to be set to the scaling parameter as well. Use self.scale for getting 4x. I will push these changes in some time.
@titu1994 I have changed the r to 4 , it can't still work.
ValueError: operands could not be broadcast together with shapes (128,64,16,16) (128,16,16,16) (128,64,16,16)
Apply node that caused the error: IncSubtensor{InplaceInc;::, ::, int64::int64, int64::int64}(Alloc.0, Subtensor{::, int64::int64}.0, Constant{0}, Constant{4}, Constant{0}, Constant{4})
Toposort index: 117
Inputs types: [TensorType(float32, 4D), TensorType(float32, 4D), Scalar(int64), Scalar(int64), Scalar(int64), Scalar(int64)]
Inputs shapes: [(128, 64, 64, 64), (128, 16, 16, 16), (), (), (), ()]
Inputs strides: [(1048576, 16384, 256, 4), (262144, 16384, 64, 4), (), (), (), ()]
Inputs values: ['not shown', 'not shown', 0, 4, 0, 4]
Outputs clients: [[IncSubtensor{InplaceInc;::, ::, int64::int64, int64::int64}(IncSubtensor{InplaceInc;::, ::, int64::int64, int64::int64}.0, Subtensor{::, int64::int64}.0, Constant{0}, Constant{4}, Constant{1}, Constant{4})]]
C:\Users\Shreyas\Image-Super-Resolution\Image-Super-Resolution>python tests.py
Using TensorFlow backend.
2018-04-10 12:19:18.301435: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Training model : ImageSuperResolutionModel
Epoch 1/250
Found 364 images.
Traceback (most recent call last):
File "tests.py", line 48, in
sr.fit(nb_epochs=250)
File "C:\Users\Shreyas\Image-Super-Resolution\Image-Super-Resolution\models.py", line 516, in fit
return super(ImageSuperResolutionModel, self).fit(batch_size, nb_epochs, save_history, history_fn)
File "C:\Users\Shreyas\Image-Super-Resolution\Image-Super-Resolution\models.py", line 122, in fit
validation_steps=val_count // batch_size + 1)
File "C:\Users\Shreyas\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\Shreyas\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 2192, in fit_generator
generator_output = next(output_generator)
File "C:\Users\Shreyas\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\utils\data_utils.py", line 793, in get
six.reraise(value.class, value, value.traceback)
File "C:\Users\Shreyas\AppData\Local\Programs\Python\Python36\lib\site-packages\six.py", line 693, in reraise
raise value
File "C:\Users\Shreyas\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\utils\data_utils.py", line 658, in _data_generator_task
generator_output = next(self._generator)
File "C:\Users\Shreyas\Image-Super-Resolution\Image-Super-Resolution\img_utils.py", line 300, in image_generator
batch_x[i] = img.transpose((2, 0, 1))
ValueError: could not broadcast input array from shape (3,32,32) into shape (3,64,64)
@titu1994 can you please help for solving this issue?
Anyone solved this?