Focus detection question
marchinram opened this issue · 3 comments
Hello,
I'm trying to understand how the dmz_focus_score_for_image function works. I read through the source and it seems that it uses the llcv_sobel3_dx_dy_c_neon function to convolve the kernel:
1, 0, -1
0, 0, 0
-1, 0, 1
with the image, then it returns the standard deviation of the output from the convolution. On iOS it seems that if the returned value is greater than 6 the focus is ok and processing continues, but if it's less than 3 it sucks and triggers an autofocus. Is this accurate? And if possible could you please explain why this works or point me in the direction of some books/articles that can explain?
As I recall from @josharian's explanation, at the time he was unable to find a published algorithm for determining focus quality, so he simply experimented with several different ideas, settling on this one as working reasonably well while still being fast.
I haven't looked at that code recently, but your description matches what I, somewhat vaguely, recall.
Dave is right. My search for an existing absolute (rather than relative) focus score was basically fruitless. I even asked SO in desperation.
I then took a bunch of recordings and experimented with different focus score calculations, with an eye towards efficiency, simplicity, and effectiveness, where effectiveness was measured both by the impact on false positives and false negatives. The selection of the technique was thus bound up in the rest of the pipeline in use.
The calculation is entirely ad hoc. I have been able to come up with no principled argument for why it should work, but objectively, it does work, and it works better than everything else I tried, including fancier algorithms. (Please don't ask for details here. At least four years have passed, and my memory is hazy at best. And the focus score was far from the most important piece of the pipeline.)
Insofar as I have been able to concoct an intuition for why this should work...a sharply focused image should have lots of sharp transitions between light and dark. These will show up after the convolution as large positive and large negative values, pushing the standard deviation up.
One known shortcoming of this focus score is that it works badly on low contrast images. However, card.io is designed to work with credit cards. If there isn't much contrast, it isn't a credit card, and we don't care. This has caused problems for people who want to use card.io just for rectangle detection--some interesting rectangles have very little contrast inside them. The recommendation for those people has simply been to disable the focus score check. Modern devices tend to focus more quickly and accurately anyway. (The iPhone 3GS had abysmal focussing ability.)
Hope that helps.
@josharian @dgoldman-ebay thanks for the explanation really appreciate it!