umyelab/LabGym

Best practice for multi-subject videos

Closed this issue · 1 comments

sarkadava commented

Hey,
I am using LabGym to detect (and later categorize) chimp behaviour. I have a few questions.

  1. I have several videos, on each video different amount of chimps. However, when I start to analyze behaviour I need to state how many chimps are on the video - this number is however not stable. For one video this will be 1, for other it could be 3. Would you suggest to run LabGym on each separately then?

  2. In those same videos, chimps are interacting which sometimes lead to one being occluded by the other (or by something else in the environment - grass, building, etc.) When manually annotating the pictures, should I annotate only the visible part of the chimp, or should I infer the whole body (for example when one chimp is behind the other)?

  3. Is it important to annotate multiple subject and always assign the same identity? Let's say I have labels chimp1, chimp2, chimp3, but I don't really care about who is who, I just want to recognize them as moving subjects and assign whatever label. Is it confusing for the LabGym if I do so?

Thanks for this tool!

yujiahu415 commented

Hi,

  1. For best results, you can run LabGym on each video separately. But you can still try to run all of them as one batch and set the number to 2 if the number ranges from 1 to 3.

  2. This depends. If the visible part is relatively small, like only a hand, or very small portion of body, when you just look at that part and cannot tell whether it's part of a chimp without help of any prior knowledge or information from other frames, then don't label it. If the visible part has dominant features of a chimp, like a face, I would suggest you label it as a "chimp". For complex environment, you need to label the occlusion scenarios as much as possible. You can have ~95% of your annotated images on the occlusion scenarios and ~5% on the well-separated scenarios.

  3. To enable the distinguishing of different identifies like chimp1, chimp2, there must be visible markers on the chimps that can be easily picked up to distinguish them, like different appearance features, body colors, or tags. Otherwise, I would suggest you just use one category, like 'chimp', for all individuals.