BrikerMan/Kashgari

[Question] NER任务,使用load_model.predict()方法进行标签预测时,若句子长度超过128会被截断,应该如何处理。

Doraemon1203-yue opened this issue · 0 comments

Environment

  • OS [e.g. Mac OS, Linux]:Windows
  • Python Version:3.6
Package                          Version
-------------------------------- ------------
absl-py                          0.15.0      
asgiref                          3.4.1       
astor                            0.8.1       
astunparse                       1.6.3       
bert4keras                       0.6.5       
cached-property                  1.5.2       
cachetools                       4.2.4       
certifi                          2023.7.22   
charset-normalizer               2.0.12      
clang                            5.0         
colorama                         0.4.5       
coreschema                       0.0.4       
Cython                           0.29.14     
dataclasses                      0.8         
flatbuffers                      1.12        
gast                             0.3.3       
gensim                           3.8.3       
google-api-core                  2.8.2       
google-auth                      1.35.0      
google-auth-oauthlib             0.4.6       
google-pasta                     0.2.0       
googleapis-common-protos         1.56.3      
grpcio                           1.32.0      
grpcio-status                    1.48.2      
h5py                             2.10.0      
idna                             3.4         
importlib-metadata               4.8.3       
importlib-resources              5.4.0       
itypes                           1.2.0       
Jinja2                           3.0.3       
jmespath                         0.10.0      
joblib                           1.1.1       
kashgari                         1.1.5       
keras                            2.6.0       
Keras-Applications               1.0.8       
keras-bert                       0.89.0      
keras-embed-sim                  0.10.0      
keras-gpt-2                      0.17.0      
keras-layer-normalization        0.16.0      
keras-multi-head                 0.29.0      
keras-pos-embd                   0.13.0      
keras-position-wise-feed-forward 0.8.0       
Keras-Preprocessing              1.1.2       
keras-self-attention             0.51.0      
keras-transformer                0.40.0      
Markdown                         3.3.7       
MarkupSafe                       2.0.1       
numpy                            1.16.4      
oauthlib                         3.2.2       
opt-einsum                       3.3.0       
packaging                        21.3        
pandas                           1.1.5       
pip                              21.3.1      
proto-plus                       1.22.3
protobuf                         3.19.6
pyasn1                           0.5.0
pyasn1-modules                   0.3.0
pycparser                        2.21
pyparsing                        3.1.1
python-dateutil                  2.8.2
pytz                             2023.3.post1
PyYAML                           6.0.1
redis                            3.5.3
regex                            2023.8.8
requests                         2.27.1
requests-oauthlib                1.3.1
rsa                              4.9
ruamel.yaml.clib                 0.2.8
scikit-learn                     0.24.2
scipy                            1.4.1
seqeval                          0.0.10
setuptools                       59.6.0
six                              1.15.0
smart-open                       6.4.0
sqlparse                         0.4.4
tensorboard                      1.14.0
tensorboard-data-server          0.6.1
tensorboard-plugin-wit           1.8.1
tensorflow                       1.14.0
tensorflow-addons                0.11.2
tensorflow-estimator             1.14.0
termcolor                        1.1.0
tf2crf                           0.1.33
threadpoolctl                    3.1.0
tqdm                             4.64.1
typeguard                        2.13.3
typing-extensions                3.7.4.3
ua-parser                        0.18.0
uritemplate                      4.1.1
urllib3                          1.26.17
Werkzeug                         2.0.3
wheel                            0.37.1
wrapt                            1.12.1
zipp                             3.6.0

Question

NER任务,使用load_model.predict()方法进行标签预测时,若句子长度超过128会被截断,应该如何处理。