specific number of pairs for training/val/testing
Closed this issue · 12 comments
Hello, thanks for sharing this repo.
As you mentioned in the paper that you randomly split 60%, 20%, 20% for training/validation/testing dataset (420, 140, 140 graphs), does it mean your pairs for training/validation/testing are 420420=176400, 140140=19600, 140*140=19600?
Also, for AIDS700nef dataset, I cannot find the grand-truth dist_mat file for the testing dataset's graphs. All Files about AIDS700nef in the /save (you shared)(https://drive.google.com/drive/folders/1Eusvi4_iOKM0AsO1LhxQFkY62kDEtuMq?usp=sharing) are about the distance matrix between training or validation graphs.
File of 'aids700nef_ged_astar_gidpair_dist_map.pickle' contains 313600 entries (560*560 pairs, only for training/validation graphs), could you share the astar grand-truth dist map for testing graphs or it is in somewhere I didn't pay attention to?
Thanks,
xiang
Hi, all.
I have a same problem.
I think there are only train/valid labels.
@yunshengb , Could you share the label for testset?
Thanks,
Junhyun Lee
All the ged_astar_gidpair_dist_map.pickles have been updated to include GEDs between test and train graphs (label for testset): https://drive.google.com/drive/folders/1Eusvi4_iOKM0AsO1LhxQFkY62kDEtuMq
Simply download them and put under /save and by loading each of these pickles you get a Python dict mapping graph id pairs to their true GED scores (raw score, unnormalized).
Please let me know if you have trouble finding or using these files. Thanks!
Thanks @yunshengb .
But the data split is not aligned with your new pickle file.
For example, in AIDS testset google drive, there are (6,30) pair.
But in AIDS pickle google drive, which is updated, there are not (6,30) pair.
Even there are (train,test) pair in your new pickle file.
Could you share exact split of your data?
I see the problem. We do not need (6,30) because both 6 and 30 are in the test set. In the paper,
The evaluation reflects the real-world scenario of graph query: For each graph in the testing set, we treat it as a query graph, and let the model compute the similarity between the query graph and every graph in the database. The database graphs are ranked according to the computed similarities to the query.
We only need the test-train GEDs as test label.
Oh, I got it. Thanks!
Then, can I just randomly select validation set within train-train set?
Yup!
Thank you!
Your answer is very helpful for me!
And I really appreciate about your fast feedback (update pickle file)!
You're welcome :)
I have a last question.
In your new pickle, there are IMDB data label with A star.
Is this label used in the paper?
According to the description, the label is determined by Beam, Hungarian, and VJ.
Yes. It is just called A star (sorry about the naming) but it is generated by Beam, Hungarian, and VJ.
Thanks, again.
And congrats your new paper accepted at IJCAI-19 ! :)
Have a nice day!
Thank you!