what's the parameter 'borders' means?
Kyle1993 opened this issue · 5 comments
Hello!
in build_model, borders control the generation of encoded image. what's the 'borders' mean? what's the different between 'no_edge' 'black' 'random' 'white' and 'image'
The border parameter describes how to pad the image during the affine transform augmentation:
black - pad with black pixels
white - pad with white pixels
random - pad each example with a random color
'image- pad with a random image (ie composite the masked transformed image on top of a random image)no_edge` - Do not perform affine transformation
@tancik Thanks for your reply, I have a confuse about the perspective transform.
in your code:
input_image --(perspective)--> input_warped --(CNN)--> residual_warped --(inverse perpective)--> residual
In my sight, residual should come from input_image, and encode_image = residual+input_image, and then get some perspective transform
Are the two method the same operation?
The CNN to encode is applied prior to the transform.
input_image --> CNN_encoder --> residual --> (perspective) --> residual_warped
We also run the following:
input_image --> (perspective) --> input_warped
This allows us to calculate the encoded image with and without the warping:
encoded_warped = input_warped + residual_warped
encoded = input + residual
@tancik I don't know if I understand correctly,
In your code, residual comes from residual_warped(by inverse perspective) and residual_warped = encoder((secret_input, input_warped))
In your reply, residual should comes from input_image directly?
Ahh, yes, it is a bit confusing. We do a prewarp to the image which in practice allows the borders to be independent of the warping. I think what ultimately matters is that some form of warping is performed, our implementation is not necessarily ideal. I think it is useful to visualize the networks inputs and outputs with tensorboard. The attached screenshot includes some annotations that are hopefully useful.
