使用BiLSTM_CRF训练几个epoch后, val_accuracy断崖式下降

[who0sy]

Environment

• ubuntu 20.04
• Python Version 3.7.0
• Kashgari 2.0.1
• tensorflow 2.4.1

Question

训练几个epoch后, val_accuracy断崖式下降, 用的是BiLSTM_CRF , 是否和CRF层有关系?

Epoch 1/20
326/326 [==============================] - 102s 273ms/step - loss: 6.5128 - accuracy: 0.9171 - val_loss: 67.1911 - val_accuracy: 0.9346
Epoch 2/20
326/326 [==============================] - 86s 263ms/step - loss: 1.0577 - accuracy: 0.9838 - val_loss: 64.9239 - val_accuracy: 0.9662
Epoch 3/20
326/326 [==============================] - 86s 263ms/step - loss: 0.7241 - accuracy: 0.9883 - val_loss: 62.8757 - val_accuracy: 0.9720
Epoch 4/20
326/326 [==============================] - 86s 263ms/step - loss: 0.5525 - accuracy: 0.9904 - val_loss: 61.1064 - val_accuracy: 0.9679
Epoch 5/20
326/326 [==============================] - 86s 263ms/step - loss: 0.4462 - accuracy: 0.9920 - val_loss: 59.4246 - val_accuracy: 0.9610
Epoch 6/20
326/326 [==============================] - 86s 264ms/step - loss: 0.3417 - accuracy: 0.9936 - val_loss: 58.0212 - val_accuracy: 0.9377
Epoch 7/20
326/326 [==============================] - 86s 264ms/step - loss: 0.2734 - accuracy: 0.9946 - val_loss: 56.6452 - val_accuracy: 0.8820
Epoch 8/20
326/326 [==============================] - 86s 264ms/step - loss: 0.2244 - accuracy: 0.9956 - val_loss: 55.5586 - val_accuracy: 0.2992
Epoch 9/20
326/326 [==============================] - 86s 263ms/step - loss: 0.1983 - accuracy: 0.9959 - val_loss: 54.3575 - val_accuracy: 0.1593
Epoch 10/20
326/326 [==============================] - 86s 264ms/step - loss: 0.1672 - accuracy: 0.9966 - val_loss: 53.4094 - val_accuracy: 0.1416
Epoch 11/20
326/326 [==============================] - 86s 264ms/step - loss: 0.1717 - accuracy: 0.9964 - val_loss: 52.4182 - val_accuracy: 0.1327
Epoch 12/20
326/326 [==============================] - 86s 264ms/step - loss: 0.1302 - accuracy: 0.9972 - val_loss: 51.5383 - val_accuracy: 0.1253
Epoch 13/20
326/326 [==============================] - 86s 264ms/step - loss: 0.1174 - accuracy: 0.9975 - val_loss: 50.7367 - val_accuracy: 0.1241
Epoch 14/20
326/326 [==============================] - 86s 264ms/step - loss: 0.1002 - accuracy: 0.9978 - val_loss: 49.8111 - val_accuracy: 0.1123
Epoch 15/20
326/326 [==============================] - 86s 264ms/step - loss: 0.0976 - accuracy: 0.9980 - val_loss: 48.9674 - val_accuracy: 0.1108
Epoch 16/20
326/326 [==============================] - 86s 265ms/step - loss: 0.0816 - accuracy: 0.9982 - val_loss: 48.2197 - val_accuracy: 0.1090
Epoch 17/20
326/326 [==============================] - 86s 265ms/step - loss: 0.0740 - accuracy: 0.9984 - val_loss: 47.5732 - val_accuracy: 0.1045
Epoch 18/20
326/326 [==============================] - 86s 264ms/step - loss: 0.0793 - accuracy: 0.9983 - val_loss: 46.5646 - val_accuracy: 0.0903
Epoch 19/20
326/326 [==============================] - 86s 264ms/step - loss: 0.0631 - accuracy: 0.9987 - val_loss: 45.8409 - val_accuracy: 0.0712
Epoch 20/20
326/326 [==============================] - 86s 264ms/step - loss: 0.0691 - accuracy: 0.9984 - val_loss: 45.1216 - val_accuracy: 0.0685

[who0sy]

添上相关训练代码:

# -*- coding: utf-8 -*-

from kashgari.corpus import ChineseDailyNerCorpus
from kashgari.embeddings import BertEmbedding
from kashgari.tasks.labeling import BiLSTM_CRF_Model
import tensorflow as tf
import os
os.environ["TF_FORCE_GPU_ALLOW_GROWTH"]="true"


train_x, train_y = ChineseDailyNerCorpus.load_data('train')
valid_x, valid_y = ChineseDailyNerCorpus.load_data('validate')
test_x, test_y = ChineseDailyNerCorpus.load_data('test')

bert_embedding = BertEmbedding("bert-base-chinese")

model = BiLSTM_CRF_Model(bert_embedding)

model.fit(train_x,
              train_y,
              x_validate=valid_x,
              y_validate=valid_y,
              epochs=20,
              batch_size=64)

model.evaluate(test_x, test_y)

model.save('saved_ner_model')

[who0sy]

从评估效果来看似乎没有太大的问题?

           precision    recall  f1-score   support

      ORG     0.9117    0.9025    0.9071      2185
      PER     0.9795    0.9726    0.9760      1864
      LOC     0.9443    0.9511    0.9477      3638

micro avg     0.9436    0.9425    0.9431      7687
macro avg     0.9436    0.9425    0.9430      7687

[Jackiexiao]

看起来是一个已知问题: #217 但没有修复

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.