Can't achive the same score in paper

Question

Can't achive the same score in paper

Opened this issue 3 years ago · 4 comments

I'm also trying to reproduce this paper. The same scores can't be achieved with the same hyperparameters. For example, I can only get 0.81 acc instead of 0.85 in the paper in SearchSnippets dataset. I think it may be the problem of data enhancement. Could you tell me whether the bertbase and roberta in the contextual argument are the original model or not, or what should I do to improve scores?

Answer 1 · 2021-08-28T02:05:09.000Z

@kirikokonoka I am also interested in this work, but I can not find the dataset to run the source code. How did you get the SearchSnippets dataset?

Answer 2 · 2021-09-02T15:22:55.000Z

@huiyegit https://github.com/rashadulrakib/short-text-clustering-enhancement/tree/master/data take a look here for the dataset

Answer 3 · 2021-11-12T06:56:48.000Z

I'm also trying to reproduce this paper. The same scores can't be achieved with the same hyperparameters. For example, I can only get 0.81 acc instead of 0.85 in the paper in SearchSnippets dataset. I think it may be the problem of data enhancement. Could you tell me whether the bertbase and roberta in the contextual argument are the original model or not, or what should I do to improve scores?

I just changed the batch_size to 250, the result was only about 0.2.I don't know what is wrong.

Answer 4 · 2022-07-05T16:49:58.000Z

Can you double check whether there is a bug in your code? Searchsnippets only has 8 classes, so accuracy of 0.2 basically indicates your model only does random cluster assignment.