How to use/include the user-defined measure of competence in deslib for DCS and DES?
sara-eb opened this issue · 3 comments
Hi,
I am using dynamic selection methods for my research that needs to use the measure of competence that I have coded instead of accuracy. Using accuracy, as the measure of competence for classifier, may not represent well the competent ones. For example, I wanted to use jaccard index from scikit-learn as a measure of competence as a start point. How to do this?
Thanks
In this case, you need to define a new dynamic selection technique and implement the estimate_competence method with the metric that is suitable for your problem.
In the case, if the technique is a DES one, you can create a new class for your method inheriting from the BaseDES class, and just implement the estimate_competence method and possibly the select too adapting to your context. A very good example that you can take as base one is the implementation of the DES-Performance method:
https://github.com/scikit-learn-contrib/DESlib/blob/master/deslib/des/des_p.py