LARS-research/SANE

bad performance

Opened this issue · 4 comments

lichaoaaron commented

Hi, after I run the code on Cora based on README.md, I got an accuracy of 85%, which is much lower than 89% as reported in the paper. Can you tell me how can I improve the performance?

hzhaoaf commented

@lichaoaaron Hi, Can you give more details about the experimental setups, including the OS, GPU info, versions of torch and pyg, etc. We need more information to check the reason.

lichaoaaron commented

Hi, thank you! The information is as follows:
GPU : TITAN RTX
CUDA Version: 10.2
python 3.7.13
pytorch 1.10.0
torch-cluster 1.6.0
torch-scatter 2.0.9
torch_geometric 2.0.4
python train_search.py --data Cora --fix_last True --epochs 20
python fine_tune.py --data Cora --fix_last True --hyper_epoch 50 --arch_filename exp_res/cora.txt

Result:
times:0, valid_Acc:89.8523941040039, test_acc:86.29629516601562
times:1, valid_Acc:89.8523941040039, test_acc:86.29629516601562
times:2, valid_Acc:89.8523941040039, test_acc:86.66666412353516
times:3, valid_Acc:90.03689575195312, test_acc:86.29629516601562
times:4, valid_Acc:89.8523941040039, test_acc:86.29629516601562

test_results_5_times:86.3704+-0.1481
finsh tunining 5 archs, saved in tuned_res/Cora_res_20220418-154824_res-20220418-154626-eps0.0-reg0.0005.pkl

hzhaoaf commented

@lichaoaaron Got it. We are conducting some experiments to check the robustness of our code. Will let you know asap.

a912289748 commented

@lichaoaaron Got it. We are conducting some experiments to check the robustness of our code. Will let you know asap.

torch _sparse version