multi-template-matching/MultiTemplateMatching-Python

Use of masks possible?

Closed this issue · 7 comments

SB2020-eye commented

If we use method = TM_CCORR_NORMED, are we able to use a mask (as one can with OpenCV's matchTemplate)?

Or are there any other ways to search for something other than a rectangle? For example:

r

rmask

LauLauThom commented

If we use method = TM_CCORR_NORMED, are we able to use a mask (as one can with OpenCV's matchTemplate)?

Currently no I don't support this option, that could be an idea for future versions but it would require quite some time-investment so I don't think it would happen soon.
Does it makes a big difference for you compared to the full square ?
I could imagine actually for letters but some example image would be great especially to test it if I implement it at some point.

Or are there any other ways to search for something other than a rectangle?

Currently you are right, it is only limited to squares.

SB2020-eye commented

Thanks for your reply. Here is an example original (reduced in size).
img-sm

I believe it makes a difference to have a mask. Letters are connected, in many cases; and other reasons, I believe, too.

LauLauThom commented

Hello, well it turns out it was not so much work 🤓😊
I have created a new branch (template-mask) that now supports masks.
I haven't released it yet because I still want to improve the docstrings but it seems to be working as expected.
I have added a new tutorial demonstrating it here.

You can already install the new version locally in your python environment directly from the code on github using this command
pip install git+https://github.com/multi-template-matching/MultiTemplateMatching-Python.git@template-mask
This will install the new version 1.6.1 ;)

I will publish this new release to pip in max a few weeks

  • 👍1
  • 🚀1
SB2020-eye commented

Thank you so much! I'm glad it wasn't too much effort.

I gave it a few tries. Like your example with the coins image, my results were very poor.

I believe I've seen different people in forums saying the tm_ccorr_normed template matching method to be worse than others. That could be it -- or part of it.

Is this mask utilized in the same way opencv uses it alongside ccorr_normed? I am just curious to know for the future, since both your results and mine were so off. (I was also curious because I had not seen the left side of the equation in cv2.threshold

theshold, mask = cv2.threshold(template,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)

used this way.)

SB2020-eye commented

Also, you mentioned filling holes in your tutorial. If interested, this is something I put together (based on someone else's work). I believe it works.

# ---------- Remove Unwanted Holes in Mask ----------

            # (adapted from https://www.learnopencv.com/filling-holes-in-an-image-using-opencv-python-c/)

            # Copy the thresholded image
            floodfillmask = mask.copy() 

            # Make mask to use for flood filling
            # NOTE: Notice the size needs to be 2 pixels larger than the image
            mh, mw = mask.shape[:2]
            maskofmask = np.zeros((mh+2, mw+2), np.uint8)

            # Floodfill from point (0, 0)
            cv2.floodFill(floodfillmask, maskofmask, (0,0), 255)

            # Invert floodfilled image
            floodfillmask_inv = cv2.bitwise_not(floodfillmask)

            # Filter out contours with less than certain area
            contours, _ = cv2.findContours(floodfillmask_inv.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
            if len(contours) == 0:
                mask_out = cv2.bitwise_or(mask, floodfillmask_inv)
            else:
                finalm = np.zeros_like(mask)
                for contour in contours:
                    carea = cv2.contourArea(contour)
                    if carea <= 140:                                                                        # $$$ ADJUSTMENT
                        cv2.drawContours(finalm, [contour], -1, 255, -1)
                mask_out = cv2.bitwise_or(mask, finalm)

(I'm pretty sure it just uses numpy and opencv. (It's part of a longer script, so I'm not 100% sure at the moment.) It assumes you already have your mask read in.)

LauLauThom commented

I gave it a few tries. Like your example with the coins image, my results were very poor.

I believe I've seen different people in forums saying the tm_ccorr_normed template matching method to be worse than others. That could be it -- or part of it.

I think it works poorly on those images because the part highlighted by the mask has a very homogeneous color (almost all black). The background is completely ignored and so just a black square in the image would perfectly match too, because it would be indeed black for the pixel covered by the mask :P
Even worse, any plain region (plain gray) will also match well because of the normalised score.

Having the background makes it more selective search: black on white, not just black.

Is this mask utilized in the same way opencv uses it alongside ccorr_normed?

Yes I just pass the mask as argument to cv2.matchTemplate

I was also curious because I had not seen the left side of the equation in cv2.threshold

the threshold function also returns the threshold value in addition to the mask image when you use OTSU or TRIANGLE method.

And about filling holes, I have seen it too but it complicates the example notebook for just a few pixels that wont make a big difference on the detection results, so I wont include it but thanks anyway !

SB2020-eye commented

Once again, thanks for your attention to all this. 👍

  • 👍1