Regarding best results obtained
Closed this issue · 4 comments
Hi there!
First of all, great work! It's been an invaluable experience for me to study your paper and your code!
Considering the results you got in your paper, the best score you got for english was 90.94%. I assume it was using the same parameters as described in this issue: #65
Still, according to this issue, it's possible to get a score of 93.86%. Is this value documented somewhere? Where can I get more insight as to how this was achieved, as opposed to the 90.94% documented in the paper?
And can you point me to the current state of the art of NER for english language? Does it use also use a Neural Networks approach, even if combined with CRF?
Thanks!
Hi,
93.86% is a result on the validation set, which is an easier dataset than the test set, this is why you have such a difference of F1 score.
Regarding the current state-of-the-art, I'm not aware of any NER open-source implementation that performs better than this one on that dataset (which does not mean that it does not exist, I actually did not really search).
Ok, I see, thanks!
Is there any reason not to consider using a greater amount of hidden units for char_lstm_dim and word_lstm_dim?
I did some experiments with the set of parameters below (calculated by a Genetic Algorithm approach), they seem to perform better for portuguese, at least. I'm still running the training of 100 epochs for english, to see how it compares.
tag_scheme=iob,
lower=True,
zeros=True,
char_dim=25,
char_lstm_dim=35,
char_bidirect=True,
word_dim=100,
word_lstm_dim=200,
word_bidirect=True,
pre_emb=Skip100,
all_emb=True,
cap_dim=1,
crf=True,
dropout=0.5,
lr_method=sgd-lr_.005
Yeah definitely this is possible. But be careful though, this also depends on your training set. CoNLL datasets are relatively small, so having a huge capacity would merely result in overfitting. But if you have a large training corpus for your data, then it's probably good to increase the number of parameters.
Thanks for the feedback Guillaume!