关于模型选择的方法合理性的疑问
JiDong-CS opened this issue · 1 comments
JiDong-CS commented
您好，我看了一下源码，对模型选择那部分有些疑惑。论文中提到，模型是根据验证集结果来选择的，但是代码实现中，只要三个度量中任意一个有提升，就更新测试集结果和对应验证集上的度量，这样做的合理性如何解释？之前有类似的做法吗？
LeePleased commented
In fact, we have tried different model selection strategies, including:
1, only considering slot f1 score;
2, only considering sentence accuracy;
3, what we do now,
but however, they all achieve similar performances.
In addition, I think it's completely fair as long as test set is not accessible while training.