CRIPAC-DIG/GCA

Cannot reproduce the results of WikiCS dataset

PatPathead opened this issue · 7 comments

PatPathead commented

Hi, thanks for reading this!

When I reproduce the results of WikiCS by the provided hyperparameters and code (nothing changed), I cannot reproduce the results shown in the paper, i.e., 78%+. I only can achieve 32%+ accuracy.

Do you have any hints on it? Is that caused by hyperparameter setting?

Thanks!

PatPathead commented

The outputs in the final several lines
(T) | Epoch=2987, loss=8.6194
(T) | Epoch=2988, loss=8.6100
(T) | Epoch=2989, loss=8.6103
(T) | Epoch=2990, loss=8.6389
(T) | Epoch=2991, loss=8.6287
(T) | Epoch=2992, loss=8.6128
(T) | Epoch=2993, loss=8.6120
(T) | Epoch=2994, loss=8.6168
(T) | Epoch=2995, loss=8.6314
(T) | Epoch=2996, loss=8.6573
(T) | Epoch=2997, loss=8.6133
(T) | Epoch=2998, loss=8.6136
(T) | Epoch=2999, loss=8.6019
(T) | Epoch=3000, loss=8.6156
(E) | Epoch=3000, avg_acc = 0.32032665610313416
0.3205062344670296

Linyxus commented

Thanks for your interest in our work! We'll check this soon.

hcmdgh commented

Hi, I also face the same problem.

brysonwx commented

Hi, I also meet with the same problem.

FinchNie commented

same problem

FinchNie commented

Using dataset ‘WikiCS(root=path)’ instead of 'WikiCS(root=path, transform=T.NormalizeFeatures())' can fix this problem.

JiaLonghao1997 commented

Hi, I also meet with the same problem.