3D Transform with UIImage Quad Calculation Methods
Closed this issue · 23 comments
In the example code you have a tab that transforms a skewed section of a given UIImage back to square (the "AGKCATransform3DWithUIImageExample"). In that example you provide a couple hard coded quads with a comment that you are working on a way to calculate those numbers for image adjustment.
Has there been any progress in this area? Or do you have any math resources that I could look at and maybe provide a pull request?
Thanks!
No progress. That part of the kit was just a spare time project. I would love to know how you solve this if you do it in objective-c or c.
I would recommend using OpenCV for this kind of stuff. I'm pretty sure they have this functionality.
Best of luck,
Håvard
Did you find any way? :)
I have been working on a way to do this, and am hoping to issue a PullRequest when I'm finished.
I've managed to generate a transform matrix, but am having some trouble applying it to an image. Trying to use -(UIImage *)[imageWithTransform:anchorPoint:], but the resulting image is not the right size for the transform, and I'm unclear on how I can make sure the image I get back is sized correctly for the image after the perspective transformation (which may not be the same dimensions as the original image).
Any suggestions? Anything I can do to help? I'm available for email/chat/whatever if need be.
Thanks.
Wow, that's awesome.
Yes, you will have a problem determining the aspect ratio of the image. I could have told you this, but I was thinking maybe OpenCV is able to determine this. I'm not sure, but it could be worth checking out.
I would be happy to assist you with setting an arbitrary size using transforms.
If you anyway are going to create a pull request and share this code maybe it would be easiest if I could have a peak at the code?
Håvard
Yeah, it seems that OpenCV does some automatic aspect ratio for you, but it may be past the point where the 3D Transform matrix is calculated. I think I found the formula to determine the correct ratio though, so we may be able to use that.
I would love to have another pair of eyes on this thing. Let me clean it up a bit, and re-apply it to my fork, then hopefully you'll be able to run it and see the correct place to resize the image.
Thanks again.
Cool!!
Your timing is impeccable. Sorry it took me a while to get here, I discovered where my earlier mistake was, and started trying to figure out aspect ratio estimation. Unfortunately, it's stumped me.
I don't know if it's the synthetically created demo image of the two warped squares, or if I've misinterpreted the aspect ratio estimation process laid out in this paper: http://research.microsoft.com/en-us/um/people/zhang/papers/tr03-39.pdf
Either way, the pull request I've submitted is set up to show the weird, but semi correct results. I changed the results image view to show as "Aspect fit" so we can see that the aspect ratio is not correct. However, if you switch it back to "Scale to fit", the resulting images seem to look about correct.
Anyway, I hope this helps, and thanks for taking a look.
Hey thanks guys. Logan, I'd be interested to see your work and potentially
use it. Can you link me to the pull request? Thanks!
-Zakk
Hey @zakkhoyt, I'm glad to hear there are more people interested in this feature. If you have any suggestions or fixes I'm happy to hear them.
To access the pull request you should be able to click where it says "Image perspective correction" above. Or a direct link to that pull request: #17
If that doesn't work for you, the pull request originated from my branch here: https://github.com/snown/AGGeometryKit/tree/image-perspective-correction
Let me know if I can help out more.
Great! Thanks Logan. I'll check it out.
On another paths, I'm very close to a solution using OpenCV. Not sure if
you're interested or can help, but there is only one small issue
(translation related). See my "answer" at the end of this thread with pics,
links, and code (I'm HCHogan):
There is a github link so you can give it a try.
Thanks
-Zakk
On Thu, Jul 24, 2014 at 10:02 AM, Logan Holmes notifications@github.com
wrote:
Hey @zakkhoyt https://github.com/zakkhoyt, I'm glad to hear there are
more people interested in this feature. If you have any suggestions or
fixes I'm happy to hear them.To access the pull request you should be able to click where it says
"Image perspective correction" above. Or a direct link to that pull
request: #17 #17If that doesn't work for you, the pull request originated from my branch
here:
https://github.com/snown/AGGeometryKit/tree/image-perspective-correctionLet me know if I can help out more.
—
Reply to this email directly or view it on GitHub
#16 (comment)
.
@zakkhoyt, cool I'll check it out. Quite a bit of my code is based on OpenCV, but written in Objective-C, so I think we're going down a similar path.
Hey thank. Guess what? I figured out the problem. Woot. Apparently the
coords were in -1.0 .. 1.0, not 0.0 .. 1.0. Applied an offset to the "to"
points and viola. Anyhow, you can check it out at the same place:
https://github.com/zakkhoyt/Quadrilateral (demo branch)
Thanks Logan.
-Zakk
On Thu, Jul 24, 2014 at 10:46 AM, Logan Holmes notifications@github.com
wrote:
@zakkhoyt https://github.com/zakkhoyt, cool I'll check it out. Quite a
bit of my code is based on OpenCV, but written in Objective-C, so I think
we're going down a similar path.—
Reply to this email directly or view it on GitHub
#16 (comment)
.
Incredible! I will look at it within the hour! 😄🏃
You wrote
I don't know if it's the synthetically created demo image of the two warped squares, or if I've misinterpreted the aspect ratio estimation process laid out in this paper: http://research.microsoft.com/en-us/um/people/zhang/papers/tr03-39.pdf
@snown I will improve the demo with better images (within a couple of weeks) 👍
Check out the develop-branch. https://github.com/hfossli/AGGeometryKit/tree/develop I have updated one of the demo controllers
I tested the -[UIImage imageWithPerspectiveCorrectionFromQuad:] method. It is behaving weirdly sometimes. I guess you'll experience the same thing if you play around with it.
Recording:
It seems safer to require the user/developer to input a desired size.
Yeah, the aspect ratio estimation has been giving me a lot of trouble. I will look at it a bit this week, but will also submit a pull request with some more options (like being able to specify the desired ending aspect ratio.
Thanks for the heads up.
Sorry for the delay in getting to this, it's been all go at work.
I was just checking into this to see what I could find, but it seems you've already resolved it. Good job making heads and tails of my code.
Is there anything else outstanding that you would like me to take a look at (other than continuing research on better aspect ratio estimation)? I can't promise how fast I'll be, but I'm very excited about this project and definitely want to help.
That is like music to my ears! 😀 Which parts of the framework would you like to be contributing to? Which parts do you take interest in?
Would it be beneficial to switch to this? https://github.com/yconst/YCMatrix/
Almost certainly. The matrix support added was something I put together to do what I needed, but by no means comprehensive.
If you want I'll take a crack at integrating YCMatrix. Do you want it as a pod dependency, subspec, or other?
Well, to me it seems like they are quite identical (AGKMatrix and YCMatrix). YCMatrix might be tested for more use cases than AGKMatrix which could be beneficial. And it could be a good thing that there are other contributors behind it (which in total means more people involved).
I would only recommend switching if we see any true value in switching. It might also be worth looking at c++ alternatives as they very well support n-sized matrixes out of the box.
We are not in a hurry and can get back to this later. 👍
@snown I've been meaning to add you ass author in the podspec, but I'm missing your email address. https://github.com/hfossli/AGGeometryKit/edit/master/AGGeometryKit.podspec