Minqi824/ADBench

The data generator fails to generate correct number of training data

ChenBaiyang opened this issue · 1 comments

ChenBaiyang commented

Hi Dear,

I found in the code, DataGenerator.generator() can not generate data properly. The parameters:

  • at_least_one_labeled does not work.
  • la and test_size are not consistent.

Thank you for your assistance.
Bryan

ChenBaiyang commented

Moreover, when I manually correct the number of training seeds, I got a result that was quite different from the reports in the paper, for example:

  • Parameter settings:
    la=0.05, test_size=0.95, at_least_one_labeled=True, model_dict = {'DevNet': DevNet, 'RF': supervised, 'CatB': supervised}

  • Results in terms of AUC:
    DevNet RF CatB
    6_cardio 0.572856 0.7334 0.847772
    25_musk 1.0 1.0 0.999278
    26_optdigits 0.974521 0.933136 0.914885
    36_speech 0.614187 0.493655 0.497312
    40_vowels 0.883842 0.805545 0.885713

  • log see attached.

Thank you very much.
Java Printing.pdf

Bryan