[Re] Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks

Question

[Re] Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks

HATON-R opened this issue 2 years ago · 37 comments

Original article:
S. Kumar, X. Zhang and J. Leskovec. Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks. In: Proceedings of the 25th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM. 2019.

PDF URL:
https://github.com/ComplexNetTSP/JODIE-RESCIENCE/blob/master/article.pdf
Metadata URL:
https://github.com/ComplexNetTSP/JODIE-RESCIENCE/blob/master/metadata.yaml
Code URL:
https://github.com/ComplexNetTSP/JODIE

Scientific domain:
Machine Learning
Programming language:
Python
Suggested editor:

Answer 1 · 2023-01-16T15:04:50.000Z

Thanks for your submission. We'll assign an editor soon.

Answer 2 · 2023-01-16T15:06:08.000Z

@gdetor @koustuvsinha Can one of you edit this submission?

Answer 3 · 2023-01-20T17:31:08.000Z

@rougier I could handle this.

Answer 4 · 2023-02-02T12:48:33.000Z

Great, thank you! I've assigned you as editor.

Answer 5 · 2023-02-13T19:57:49.000Z

Hi @ogrisel, would you be able to review this submission?

Answer 6 · 2023-02-13T20:00:53.000Z

@rougier Could I assign as reviewer someone off the reviewer's list?

Answer 7 · 2023-02-14T13:08:05.000Z

Yes of course. If they accept and want to appear in the board, just tell me. You can also ask all reviewers at once using the @ ReScience / reviewers notification (without space, I don't want to broadcast here)

Answer 8 · 2023-02-14T18:15:14.000Z

Hi @ghost-nn-machine would you be able to review this submission?

Answer 9 · 2023-03-07T02:14:12.000Z

Hi @benureau Could you handle this review?

Answer 10 · 2023-03-28T02:25:22.000Z

Hi @koustuvsinha would you be willing to review this submission?

Answer 11 · 2023-03-28T02:26:24.000Z

Hi @neuronalX could you handle the review of this submission?

Answer 12 · 2023-05-02T08:11:11.000Z

Hi @gdetor, thank you for the offer, but I am already too busy for the following month.

Answer 13 · 2023-05-23T06:41:32.000Z

Hey @gdetor, I can handle this.

Answer 14 · 2023-07-11T03:04:10.000Z

Hi @damiendr Would you be available to review this submission?

Answer 15 · 2023-07-24T21:28:41.000Z

HI @hkashyap
Could you handle this review?
@ghost-nn-machine Gentle reminder

Answer 16 · 2023-07-25T16:41:14.000Z

@gdetor I can review this submission.

Answer 17 · 2023-08-02T19:48:08.000Z

Thank you @hkashyap I'll assign you as a reviewer.

Answer 18 · 2023-10-15T16:37:32.000Z

Hi @hkashyap and @ghost-nn-machine Any updates?

Answer 19 · 2023-10-16T05:56:32.000Z

@gdetor I will need more time, I plan to submit the review by 10/30.

Answer 20 · 2023-11-16T15:13:34.000Z

Gentle reminder.

Answer 21 · 2023-12-07T17:40:05.000Z

Hi @hkashyap @ghost-nn-machine Any progress?

Answer 22 · 2024-01-25T13:15:19.000Z

I trust this message finds you well. I am writing to request an update on the manuscript I submitted for review over a year ago today. Might you be able to provide me with some insight into the current status of the review process?

Perhaps we could consider the revisions as a symbolic birthday present for my article?

Answer 23 · 2024-02-05T14:21:33.000Z

@HATON-R Very sorry for being so late in the review. I'll try to make things move forward.

Answer 24 · 2024-02-05T14:22:16.000Z

@ReScience/reviewers Help needed for reviewing a paper machine learning/Python ! See #70

Answer 25 · 2024-02-05T14:22:57.000Z

@HATON-R Don't hesitate to remind us here we're late. We have not yet an automated process for tracking submission (but we'll soon have)

Answer 26 · 2024-02-05T20:42:42.000Z

I can review this as well. @rougier

Answer 27 · 2024-02-06T08:05:30.000Z

@HaoZeke Thank you! You can start the review then. If you can do it in less than two weeks that would be wonderful.

Answer 28 · 2024-02-06T08:06:20.000Z

@ghost-nn-machine Are you still available to do the review?

Answer 29 · 2024-02-06T12:22:23.000Z

@HaoZeke Thank you! You can start the review then. If you can do it in less than two weeks that would be wonderful.

Sure, I'll try to get it done this weekend.

Answer 30 · 2024-02-12T17:25:43.000Z

@hkashyap Can you update us on your review (just tell us if you can't do it such that we start looking for another reviewer)

Answer 31 · 2024-05-08T18:19:30.000Z

@hkashyap @HaoZeke Any progress?

Answer 32 · 2024-05-27T13:28:57.000Z

@HaoZeke Any update?

Answer 33 · 2024-08-27T22:04:35.000Z

@HaoZeke @hkashyap Any updates?

Answer 34 · 2024-08-31T05:40:55.000Z

@HATON-R @gdetor Maybe we need to find other reviewers. I can make a review and maybe @gdetor you can do the second one. Let's give us two weeks?

Answer 35 · 2024-09-03T20:20:33.000Z

@rougier That works for me

Answer 36 · 2024-09-03T22:19:13.000Z

@HATON-R @rougier Here is my review.

Overall, the work shows that one can replicate the main results of the original. Moreover, the authors go the extra mile and show how the model's basic hyperparameters affect its performance.

Text

Although the authors refer to slightly different conclusions than those of the original article, they do not mention them in the summary.
The mathematical symbol for the Haramard product is $\odot$.
In the t-batch section, the authors mention that a batch cannot contain the same entity several times. Please be more specific.
The two paragraphs "When we want to predict ... cross-entropy expressed in the following equation:" on page 7 can be merged. The authors repeat themselves.
One can mistake the number of classes C for the number of t-batches in algorithm 1.
In the first paragraph of Section 3.5, the authors repeat themselves. Please rephrase that paragraph.
When presenting their results, it might be better if the authors used the same format for the tables as in the original article. They can add their results in bold font and as a row in the table.
The authors claim their results are marginally worse than the ones reported in the original work. However, it's not marginal because, for instance, the RRN performs better on the Reddit and Wikipedia datasets.

Source Code

Unfortunately, due to Ray incompatibility, I couldn't run the code and verify the results. The authors have used an older version of Ray, so please update it or impose the exact version in the requirements.txt.

Please add a LICENSE file
The code is not documented; please add some essential documentation, at least in the main components of the source code
The source code should follow Python's PEP-8 protocol. The autopep-8 tool can help you make your scripts comply.
There are imported packages that are not used in the code (e.g., ray tune, ray.air in file train.py)
The code does not run on Pytorch 2.0 (the authors should fix that since many people use that version instead of the 1.10)
Did the authors try to use any specialized search algorithm with Ray Tune, such as Optuna, or do they rely only on Ray's default algorithms?
The requirement for Torch is broken. Use the -i link and then the name of the package.

Answer 37 · 2024-10-23T21:07:12.000Z

@rougier Gentle reminder