Assertion triggered for certain image sizes
SlobodanCv opened this issue · 11 comments
Thank you for a very useful contribution- this is the only one I found that implements the phase correlation on log-polar images in C++.
I was trying to use it to emulate Matlab function imregcorr(). There are some obvious differences- this code accepts only the images that are the same size, and does not return the peak correlation coefficient. Any pointers how to resolve this would be greatly appreciated.
I found that the code runs into problems for certain image sizes. For example, if you resize im0 and im1 provided with the source code to be 897x1151 (width x height), function Eigen::MatrixXd ImageDFT::neighbourhood() will try to use a negative index in a wrap-around section of the code, causing an assertion eigen_assert(row >= 0 && row < rows() && col >= 0 && col < cols());
Thanks for tracking down that error; it should be fixed now.
- images of different sizes: you're right, at the time I was working with videos so all frames had the same size. I haven't looked into where we'd have to make the change to allow images of different sizes. If you figure it out, feel free to send a PR. I will also look into it (later this year) when I get some time.
- I think the peak correlation coefficient refers to the value of the CPS at the peak? The approximate value would be the value of
absat the index[approx_row, approx_col](see here). Since it's computing the sub-pixel peak, you'd need to do some interpolation similar to how the centre of mass is computed.
Btw, there are two other C++ implementations that I've seen, which use OpenCV functions for logpolar transformation and phase correlation. Here and here
Thank you for a quick reply and references, it's very helpful.
I'm afraid that there are still some issues- the assertion is not triggered after I applied your fix, but the next function call after neighborhood() returns, ImageDFT::getCentreOfMass(const Eigen::MatrixXd &f1, double &row, double &col) will place NaN values in row and col, and they will propagate further to rotation and scale parameters. To check if this is because of the image content (thinking that maybe there's no solution for a transformation matrix), I tried different test images. I used the same size 897x1151 pixels, but just the square on a dark background that I translated, rotated and scaled a bit. The same issue still persists- I get NaN for rotation and scale parameters.
Could you share the images that you're using? I tried resizing those images to 897x1151 and did not get the initial error that you reported.
I'm sorry, I have hard time figuring out what's going on. When I tested your fix, I was running in the Debug mode, and I saw NaN values coming out, as I described. In Release mode I don't get NaN values, but final transformation parameters are off by a lot. Scale 0.99 vs 1.2 used, rotation 0.03 vs 5.0 used etc. I included the images I used for testing. If I use test images in the size you provided with the same content as in my test (just a gray rectangle that is translated, rotated, and scaled), I get perfect results in both Debug and Release mode.
Thanks for that info and the sample images. If there are differences between the debug and release versions (especially related to NaN/unexpected values), that probably means there is a memory leak somewhere :(. I didn't see NaN values with debug or release, so I'm not sure where to start debugging.
The fftShift function was previously only working for even image sizes, so it's now been updated to work for odd images sizes as well. There seems to still be some issue with odd image sizes though, which I haven't figured out yet. I'll update the issue if I find the problem.
Thank you for the latest fix. Changing fftShift function actually fixed NaN issues for the Debug version- now the results are consistent between the Debug and Release versions.
As you mentioned, there is still an issue with bad transformation parameters for some images. I'll experiment a bit to see if i can figure out the type of images that cause this to happen- it happens for even image size as well.
Interestingly enough, the problem seems to appear only in artificial test images, as ones shown in one of my older comments. Typically rotation estimate fails, and gives values like -179 or something like this. This error than propagates. After the fixes you did, your code works well for natural scenes and images for both odd and even sizes. I tested these artificial images with other code (MATLAB and other GitHub implementations you shared with me), and the results are also pretty poor. It seems that the issue relates to how the log-polar phase correlation algorithm performs for this type of images, not in your implementation. I'm closing this issue. Thanks again for your support!
Thanks for finding those issues with the code!
While testing I found that there is still a problem with some images. For example, I scaled the images in docs to 1280x720, and it produces incorrect results for the scaled image. You're right that it has something to do with the first phase (i.e. phase correlation on the log-polar image). So anyway, I will look into it at some point and update this issue if I find something.
Could it be because you used different scale factors for x and y (scales 3 and 4)? I'm not sure that your second image would still be a result of similarity transform of the first image if you apply this type of scaling to both of them.
Do you mean somewhere in the code, or for the input images? I'm not sure where you mean (if you mean in the code).