umyelab/LabGym

issue with Test categorizers

fengaccoumt opened this issue · 15 comments

fengaccoumt commented

Good afternoon! Thanks for a wonderful piece of programming.

I get trained categorizer now. But when I want use the ground up example to test trained categorizer, I can't get the result. In terminal it like this:
terminal_test trained categorizer

This is my trained categorizer report in terminal :
image

This is my ground up example‘s name:
image

I want to know how to solution this question?

Thanks you a lot!

yujiahu415 commented

Hi,

  1. You need to have at least two categories to enable the training / testing. It seems you only used one behavior category for testing.
  2. You need to make the behavior name in testing dataset matches that in the Categorizer. The behavior name in your testing dataset is 'limptail_hindleg' while in your terminal it says 'behavior name in the Categorizer is hindleg'.
fengaccoumt commented

Thanks for your answers.

But the problem also remain. This time I use four categories and harmonize the behavior name of Categorizer and testing dataset , but it also show unused behavior names in the Categorizer in terminal like this.
image

This is my behavior name for training Categorizer:
image

This is my ground up example's behavior name:
image

Thanks you a lot!

yujiahu415 commented

Behavior names should be English letters (can contain "-" or " "). But please do not use numbers, special characters such as "#", or "_". Let me know if you still have the issue when you use English letters.

fengaccoumt commented

Thanks for your answers.

Sorry, I still have the issue when I use English letters, the error like this in terminal.
image

Thanks you a lot!

fengaccoumt commented

The hole error in terminal is like this.
image

yujiahu415 commented

The error message indicated that one image file (you can find the name in the error message) in the "aa" folder was either broken, or has a path that could not be read properly. Please check that file, either replace or remove that file together with its associated "animation" file.

fengaccoumt commented

Thanks for your answers.

But the new error come out. It like this in terminal.
image

Thanks you a lot!

fengaccoumt commented

Thanks for your help, I get training report now. But the value of precision, recall and f1-score is very low, how can I get the accuracy good Categorizer?

image

Thanks you a lot!

yujiahu415 commented

Increase the number of training examples and sort them well. Typically around 100 pairs for each behavior category. You may also increase the number of your testing examples, that will give the test more statistical power. Currently there's only three examples for each category in your testing data.

fengaccoumt commented

Thank you very much for your help, now I have been able to get the test report, and increase the accuracy to 0.91.
image

yujiahu415 commented

Thanks for letting me know this. Great to hear that!

fengaccoumt commented
yujiahu415 commented

Hello! I would like to ask if using different frames of behavior examples to train categorizers will affect the performance of the model? Thanks for your reply. Yujia Hu @.> 于2024年2月7日周三 10:08写道：
…
Thanks for letting me know this. Great to hear that! — Reply to this email directly, view it on GitHub <#89 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFS3XYKACJWE2O64ECBI5PLYSLHYNAVCNFSM6AAAAABCSJRRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA3DENBXHE . You are receiving this because you authored the thread.Message ID: @.>

What do you mean by "using different frames of behavior example"? Did you mean using behavior examples of different fps? Or just different behavior examples? How the trained Categorizer performs depends on how you select and sort the behavior examples.

fengaccoumt commented
yujiahu415 commented

The more examples for testing, the more statistical power of the testing. In the other hand, the more examples for training, the better performance of the training. Generally speaking, 100 pairs of examples for each behavior category for testing should give you sufficient statistical power, and all the resting examples can be used for training.