found two bugs that could cause your inferior performance than original paper

Question

found two bugs that could cause your inferior performance than original paper

jind11 opened this issue 7 years ago · 17 comments

Hi, I have carefully read your code and found two bugs that could potentially cause your inferior performance compared with the original paper. 1. in the new_convolution function, there is missing use of self.tanh() as the activation after the convolution layer; 2. in the original paper, the convolution kernel is 1 since the input is already trigram, so there is no need to use kernel size of 3 in the new_convolution function. If you have doubt on my comments, welcome to discuss with me, thanks!

Answer 1 · 2017-11-27T13:24:09.000Z

ohh.i think the conv function include tanh activation. it is an interesting understanding on kernel. i think you may right. i will reconsider the kernel implementation. thanks for your discussion. keep connection

Answer 2 · 2017-11-27T15:45:41.000Z

hi, thanks for the quick response. But I am sure the conv function at Pytorch does not include any activation function.

Answer 3 · 2017-11-27T23:01:08.000Z

i had read paper second time and found that kernel should be 1. cnn should not comprise activation, i searched that. could we build some new connection? you helped a lot. thank you

Answer 4 · 2017-11-29T04:03:01.000Z

sure! my email is jindi15@mit.edu. And I also have some other revisions to your code, if you want, I can sent it to you for references.

Answer 5 · 2017-11-30T01:49:34.000Z

copy that. so nice to you.

Answer 6 · 2017-12-03T20:05:27.000Z

hi, what is your email? I have some other questions for the algorithm understanding. Of if you are in US, we can have a phone call. My phone number is 617-710-6221

Answer 7 · 2017-12-03T22:36:35.000Z

sorry, i had sent an email to you 4 days ago. maybe my gmail can not work in the wall. oh, i am in China by the way. the policy of vpn reduce our space further.😂😂😂
my working email is dgai_ruc@aliyun.com.

Answer 8 · 2017-12-05T23:04:55.000Z

hi, sent you an email to dgai_ruc@aliyun.com but did not get reply. my wechat is jindi930617, feel free to add me if you want

Answer 9 · 2017-12-08T13:18:41.000Z

Hi everybody,

I cannot reproduce the results at all (~25%). What were your final macro F1-score ?

Best

Answer 10 · 2017-12-21T02:22:26.000Z

@Diego999 hi. maybe you can replace the paper

Answer 11 · 2017-12-21T04:46:54.000Z

@lawlietAi hi, I have sent you a email for the same confusion. Looking forward to your reply

Answer 12 · 2017-12-21T05:28:59.000Z

@ShomyLiu thx for your discussion. i'd replied you minutes ago

Answer 13 · 2018-01-19T05:11:01.000Z

Hi, any progress? What's your F1 score now?

Answer 14 · 2018-09-27T04:12:09.000Z

Hi, did you reproduce the performance declared in the original paper? I implemented it with TF, but the performance is much lower.

Answer 15 · 2018-09-27T04:26:27.000Z

no, I can never replicate the performance in that paper. my performance is not good, around 82.7%

Answer 16 · 2018-10-08T15:15:35.000Z

no, I can never replicate the performance in that paper. my performance is not good, around 82.7%

why my accuracy of model just is 25% ?

Answer 17 · 2018-10-08T15:17:39.000Z

Hi everybody,

I cannot reproduce the results at all (~25%). What were your final macro F1-score ?

Best

Have you found the problem of accuracy with 25%?