Rough segmentation with many overlapping polygons lead to freeze
Closed this issue · 19 comments
Hi, first let me thank you for the great tool! :)
We have however found an annoying bug, where the tool freezes if polygons touch. As this happens only under unknown circumstances I hope you might be able to help us out.
The test ran on a well equipped PC (i7, 64GB RAM, Ubuntu 18.04, Firefox/Chromium). The problem is still there after pulling the latest version.
Attached you will find a video (2020-01-14_hitachi_labeltool_freeze_bug_short.mp4) demonstrating the issue. If we draw the same polygons with some space, i.e., not connecting to other polygons, we have no issues. But the label is imprecise of course.
Maybe you can hint me to the point where I can debug the issue, it happens during polygon creation. We are using a dockered version however, which might complicate things a bit.
Best
Georg
Thanks Georg :)
Looking at your video, I can't figure out what's going on.
The best thing you can do to help me spotting the origin of the bug is to open the debugger of the browser and see if something happen on the console when you hit the problem.
You may want to prefer Chromium, I noticed it's more explicit in some cases when something wrong happens.
Best
Damien
Hi Damien,
thx for the quick reply. Chromium performance logging provides indeed more details. It seems the recursive findPathRec is taking very long (~900ms per call). To set this in context, in the sample we have 27 polygons with a total of 848 points. The polygon that is "touched" hast 63 points.
Any idea how to proceed?
No idea...since the number of polygons/points is low, it's not supposed to hang. If you have nothing on the console it would just be a matter of performance but why? I don't know but it's not about your machine which is very strong.
I guess you run the app on docker and access it from an outside browser on the same machine. You can try to label an image from another machine and/or try to run the app without docker.
If possible you can also share an actual image to let me reproduce.
Took me quite some time to setup and test but I have composed a repo containing:
- dockerization of the tool
- sample to reproduce the issue
The Readme should contain all necessary steps in quite some detail. Would be happy if it helps to resolve the issue.
Thanks for this amazing report! It works so well that...I do not reproduce your bug :)
I deployed the app exactly as described in your repo in a fresh Ubuntu virtual machine and I'm able to draw contiguous polygons on your image without experimenting any UI freeze using stock Firefox on the virtual machine. However I haven't be able to use sample.json, the tool crash.
Maybe you should duplicate your image and start a labeling from scratch.
When I will have some time, I will try to integrate some docker stuff in the project by re-using your instructions. Thanks again for that, and sorry but I do not know what's wrong on your side.
Thanks for the fast feedback, glad it worked for you and might help to add docker functionality to your tool.
However I haven't be able to use sample.json, the tool crash.
Well, this is exactly the problem. We don't experience crashes regularly, only at certain (unknown!) circumstances. This is why I provided one of the samples that leads to problems. It has nothing to do with the json format or content, as this was exported after drawing with the tool.
Any further tips for debugging? Something with the findPathRect must go wrong under certain constellations of polygons...
The tool crashes because the format of the data imported using mongoimport is not correct.
Imported sample.json:
Regular format:
You can try to use mongodump to share the broken labeling.
Ah sorry, misunderstanding. Will get back to you with the new export as soon as I can.
What version of mongodb are you running? I did not experience problems using mongoimport, maybe our versions differ (v4.2.2 here).
Yes it is a different version, apt installed Mongo 3.6.3 by default...
I tried to install Mongo 4+, it failed and my virtual machine is now in a fully broken state regarding mongo apt packages.
I'm sorry but it's pretty long to setup again the full stack you described or investigate how to fix apt packages. Running the tool using docker/standalone mongo is out of the scope of this open source project and I doubt the problem is about findPathRect.
Hope you understand, feel free to provide a fully automatic install script that runs on a fresh Ubuntu installation to let me continue investigations on your very specific issue. You can also reach me on Linkedin if you are interested in a paid support service ;)
Best
Damien
I exported the sample lables again with mongodump instead of some GUI, I think now the format should be ok. Can you give it another try please?
Sorry I can't, Mongo is now broken and no longer runs on my virtual machine.
I didn't reload the page and did not see your new comment but only read it afterwards.
I'm on it to build a bash install script for installation and debugging on a fresh Ubuntu 18.04 virtual machine.
I still think it is related to findPathRect. There must be some constellation that leads to a lot of expensive point checks. As you experienced, the tool runs fine within the docker and with external MongoDB, it is only sometimes that the freeze happens.
Finally I have had some time to compile a set of commands into two scripts (setup_scripts.zip) to get the buggy sample set up.
The first one does all the installation and starts the labeltool. The second one should be run once the labeltool is up to import the sample labels to the MongoDB of the running instance.
As said before, I used the Ubuntu 18.04.3 image from osboxes. The VM was with 8GB RAM and 4 CPUs, less should also work but I didn't want to risk the VM config to cause problems.
After initial startup I had to disable the automatic updates (Software & Updates -> Updates). Otherwise the script would not work because of the lock. I also disabled screen lock as the procedure might take some time.
I also noticed, if you wait long enough the polygon that causes the freezing will finish and add to the others. Still undesired as this slows down the labeling dramatically. :/
I now reproduce your issue which looks to be related to the way you are using the tool. The basic idea behind the tool is to draw perfectly contiguous polygons, because a pixel is not supposed to belong to multiple classes. Making a rough segmentation with many overlapping polygons lead to abnormal recursion of findPathRect. Solving this issue would probably need to rethink the way it' done. One possible mitigation would be to comment what is related to pixel snapping to get a completely free-hand drawing tool. Thanks for your very complete report on this actual problem which is unfortunately hard to solve.
Hello Damien,
thank you very much for your effort to debug my problem. It really helps to understand the cause of the issue. I will rethink the labeling strategy and how to solve this problem on the user side.
many overlapping polygons on the same layer
Does this mean if we move overlapping polygons to different layers it should be no problem?
No...my bad... Pixel snapping also happens between layers.
Ok, so possible whick solutions would be:
- draw no or very few(?) overlapping polygons
- deactivate the snapping where the
findPathRectis used
If the latter is possible, a hint where to deactivate the snapping would be highly appreciated. ;)
@edit: Would it be sufficient to comment the line 162 in the SsePolygonTool.js file?
Well, it should be pretty easy :) Just comment the content of findPathRect to make it an empty function.