The results do not match the description in Readme.md
huayangcs opened this issue · 0 comments
huayangcs commented
hello, thanks for this amazing benchmark work. However, when I reproduced Pgexplainer's results using the data you provided, the trained model you provided, and the pgexplainer model you provided, I found that the results were significantly different from the ones you provided in Readme.md. May I ask the possible reasons for this result? Thanks for your help.
DATASETS=ba_2motifs
python -m benchmarks.xgraph.pgexplainer_edges datasets=$DATASETS explainers=pgexplainer
results are :
Fidelity: 0.4920
Fidelity_inv: 0.4253
Sparsity: 0.5066
Accuracy: 0.04254115393283751
Stability: nan
after I change the config.explainers.sparsity = 0.8, the results are
Fidelity: 0.3911
Fidelity_inv: 0.4889
Sparsity: 0.8089
Accuracy: 0.04254115393283751
Stability: nan