How to Predict NER
Closed this issue · 4 comments
lizaigaoge550 commented
There only have a command that told us how to train models. Could you provide some predict commands, like how do i use the trained ner model to get predict tags? Thank you very much.
kyleclo commented
We use the AllenNLP framework for our models, so will defer you to their tutorial for using the model for prediction. https://github.com/allenai/allennlp/blob/master/tutorials/getting_started/predicting_paper_venues/predicting_paper_venues_pt2.md#making-predictions
dmarx commented
Would you please consider offering sample code for making predictions? I trained an NER model per the README instructions, and I'm unable to even get predictions on the test set. The error looks like it's coming from spacy, which is strange to me considering I'm using the out-of-the-box dataset reader on pre-formatted data that was successfully parsed during model training. Here's what I'm trying:
python -m allennlp.run predict ./modeltest0_ner/model.tar.gz data/ner/bc5cdr/test.txt --include-package scibert --use-dataset-reader
I've also tried adding --predictor sentence-tagger to the end of that command. With or without that explicit predictor invocation, I get the same result:
2019-08-16 00:37:35,218 - INFO - pytorch_pretrained_bert.modeling - Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex .
/usr/local/lib/python3.6/dist-packages/sklearn/utils/linear_assignment_.py:21: DeprecationWarning: The linear_assignment_ module is deprecated in 0.21 and will be removed from 0.23. Use scipy.optimize.linear_sum_assignment instead.
DeprecationWarning)
2019-08-16 00:37:36,348 - INFO - allennlp.models.archival - loading archive file ./modeltest0_ner/model.tar.gz
2019-08-16 00:37:36,349 - INFO - allennlp.models.archival - extracting archive file ./modeltest0_ner/model.tar.gz to temp dir /tmp/tmpmyhhl9ut
2019-08-16 00:37:40,593 - INFO - allennlp.common.params - type = default
2019-08-16 00:37:40,593 - INFO - allennlp.data.vocabulary - Loading token dictionary from /tmp/tmpmyhhl9ut/vocabulary.
2019-08-16 00:37:40,595 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.models.model.Model'> from params {'calculate_span_f1': True, 'constrain_crf_decoding': True, 'dropout': 0.5, 'encoder': {'bidirectional': True, 'dropout': 0.5, 'hidden_size': 200, 'input_size': 896, 'num_layers': 2, 'type': 'lstm'}, 'include_start_end_transitions': False, 'label_encoding': 'BIOUL', 'text_field_embedder': {'allow_unmatched_keys': True, 'embedder_to_indexer_map': {'bert': ['bert', 'bert-offsets'], 'token_characters': ['token_characters']}, 'token_embedders': {'bert': {'pretrained_model': 'scibert_scivocab_uncased/weights.tar.gz', 'type': 'bert-pretrained'}, 'token_characters': {'embedding': {'embedding_dim': 16}, 'encoder': {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128, 'type': 'cnn'}, 'type': 'character_encoding'}}}, 'type': 'crf_tagger'} and extras {'vocab'}
2019-08-16 00:37:40,596 - INFO - allennlp.common.params - model.type = crf_tagger
2019-08-16 00:37:40,596 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.models.crf_tagger.CrfTagger'> from params {'calculate_span_f1': True, 'constrain_crf_decoding': True, 'dropout': 0.5, 'encoder': {'bidirectional': True, 'dropout': 0.5, 'hidden_size': 200, 'input_size': 896, 'num_layers': 2, 'type': 'lstm'}, 'include_start_end_transitions': False, 'label_encoding': 'BIOUL', 'text_field_embedder': {'allow_unmatched_keys': True, 'embedder_to_indexer_map': {'bert': ['bert', 'bert-offsets'], 'token_characters': ['token_characters']}, 'token_embedders': {'bert': {'pretrained_model': 'scibert_scivocab_uncased/weights.tar.gz', 'type': 'bert-pretrained'}, 'token_characters': {'embedding': {'embedding_dim': 16}, 'encoder': {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128, 'type': 'cnn'}, 'type': 'character_encoding'}}}} and extras {'vocab'}
2019-08-16 00:37:40,596 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.text_field_embedders.text_field_embedder.TextFieldEmbedder'> from params {'allow_unmatched_keys': True, 'embedder_to_indexer_map': {'bert': ['bert', 'bert-offsets'], 'token_characters': ['token_characters']}, 'token_embedders': {'bert': {'pretrained_model': 'scibert_scivocab_uncased/weights.tar.gz', 'type': 'bert-pretrained'}, 'token_characters': {'embedding': {'embedding_dim': 16}, 'encoder': {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128, 'type': 'cnn'}, 'type': 'character_encoding'}}} and extras {'vocab'}
2019-08-16 00:37:40,596 - INFO - allennlp.common.params - model.text_field_embedder.type = basic
2019-08-16 00:37:40,597 - INFO - allennlp.common.params - model.text_field_embedder.allow_unmatched_keys = True
2019-08-16 00:37:40,597 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.token_embedders.token_embedder.TokenEmbedder'> from params {'pretrained_model': 'scibert_scivocab_uncased/weights.tar.gz', 'type': 'bert-pretrained'} and extras {'vocab'}
2019-08-16 00:37:40,597 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.bert.type = bert-pretrained
2019-08-16 00:37:40,597 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.token_embedders.bert_token_embedder.PretrainedBertEmbedder'> from params {'pretrained_model': 'scibert_scivocab_uncased/weights.tar.gz'} and extras {'vocab'}
2019-08-16 00:37:40,597 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.bert.pretrained_model = scibert_scivocab_uncased/weights.tar.gz
2019-08-16 00:37:40,597 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.bert.requires_grad = none
2019-08-16 00:37:40,597 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.bert.top_layer_only = False
2019-08-16 00:37:40,597 - INFO - pytorch_pretrained_bert.modeling - loading archive file scibert_scivocab_uncased/weights.tar.gz
2019-08-16 00:37:40,598 - INFO - pytorch_pretrained_bert.modeling - extracting archive file scibert_scivocab_uncased/weights.tar.gz to temp dir /tmp/tmpv09jz3vo
2019-08-16 00:37:44,759 - INFO - pytorch_pretrained_bert.modeling - Model config {
"attention_probs_dropout_prob": 0.1,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"initializer_range": 0.02,
"intermediate_size": 3072,
"max_position_embeddings": 512,
"num_attention_heads": 12,
"num_hidden_layers": 12,
"type_vocab_size": 2,
"vocab_size": 31090
}
2019-08-16 00:37:47,390 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.token_embedders.token_embedder.TokenEmbedder'> from params {'embedding': {'embedding_dim': 16}, 'encoder': {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128, 'type': 'cnn'}, 'type': 'character_encoding'} and extras {'vocab'}
2019-08-16 00:37:47,390 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.type = character_encoding
2019-08-16 00:37:47,390 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.num_embeddings = None
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.vocab_namespace = token_characters
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.embedding_dim = 16
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.pretrained_file = None
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.projection_dim = None
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.trainable = True
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.padding_index = None
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.max_norm = None
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.norm_type = 2.0
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.scale_grad_by_freq = False
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.embedding.sparse = False
2019-08-16 00:37:47,391 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.seq2vec_encoders.seq2vec_encoder.Seq2VecEncoder'> from params {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128, 'type': 'cnn'} and extras set()
2019-08-16 00:37:47,391 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.type = cnn
2019-08-16 00:37:47,392 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.seq2vec_encoders.cnn_encoder.CnnEncoder'> from params {'conv_layer_activation': 'relu', 'embedding_dim': 16, 'ngram_filter_sizes': [3], 'num_filters': 128} and extras set()
2019-08-16 00:37:47,392 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.embedding_dim = 16
2019-08-16 00:37:47,392 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.num_filters = 128
2019-08-16 00:37:47,392 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.ngram_filter_sizes = [3]
2019-08-16 00:37:47,392 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.conv_layer_activation = relu
2019-08-16 00:37:47,392 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.encoder.output_dim = None
2019-08-16 00:37:47,395 - INFO - allennlp.common.params - model.text_field_embedder.token_embedders.token_characters.dropout = 0.0
2019-08-16 00:37:47,395 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.modules.seq2seq_encoders.seq2seq_encoder.Seq2SeqEncoder'> from params {'bidirectional': True, 'dropout': 0.5, 'hidden_size': 200, 'input_size': 896, 'num_layers': 2, 'type': 'lstm'} and extras {'vocab'}
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.type = lstm
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.batch_first = True
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.stateful = False
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - CURRENTLY DEFINED PARAMETERS:
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.bidirectional = True
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.dropout = 0.5
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.hidden_size = 200
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.input_size = 896
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.num_layers = 2
2019-08-16 00:37:47,396 - INFO - allennlp.common.params - model.encoder.batch_first = True
2019-08-16 00:37:47,421 - INFO - allennlp.common.params - model.label_namespace = labels
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.label_encoding = BIOUL
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.include_start_end_transitions = False
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.constrain_crf_decoding = True
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.calculate_span_f1 = True
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.dropout = 0.5
2019-08-16 00:37:47,422 - INFO - allennlp.common.params - model.verbose_metrics = False
2019-08-16 00:37:47,423 - INFO - allennlp.nn.initializers - Initializing parameters
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - Done initializing parameters; the following parameters are using their default initialization from their code
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - crf._constraint_mask
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - crf.transitions
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_hh_l0
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_hh_l0_reverse
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_hh_l1
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_hh_l1_reverse
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_ih_l0
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_ih_l0_reverse
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_ih_l1
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.bias_ih_l1_reverse
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.weight_hh_l0
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.weight_hh_l0_reverse
2019-08-16 00:37:47,424 - INFO - allennlp.nn.initializers - encoder._module.weight_hh_l1
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - encoder._module.weight_hh_l1_reverse
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - encoder._module.weight_ih_l0
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - encoder._module.weight_ih_l0_reverse
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - encoder._module.weight_ih_l1
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - encoder._module.weight_ih_l1_reverse
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - tag_projection_layer._module.bias
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - tag_projection_layer._module.weight
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.gamma
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.0
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.1
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.10
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.11
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.2
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.3
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.4
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.5
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.6
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.7
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.8
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert._scalar_mix.scalar_parameters.9
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.embeddings.LayerNorm.bias
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.embeddings.LayerNorm.weight
2019-08-16 00:37:47,425 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.embeddings.position_embeddings.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.embeddings.token_type_embeddings.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.embeddings.word_embeddings.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.output.LayerNorm.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.output.LayerNorm.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.output.dense.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.output.dense.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.key.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.key.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.query.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.query.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.value.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.attention.self.value.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.intermediate.dense.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.intermediate.dense.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.output.LayerNorm.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.output.LayerNorm.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.output.dense.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.0.output.dense.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.output.LayerNorm.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.output.LayerNorm.weight
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.output.dense.bias
2019-08-16 00:37:47,426 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.output.dense.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.key.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.key.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.query.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.query.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.value.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.attention.self.value.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.intermediate.dense.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.intermediate.dense.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.output.LayerNorm.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.output.LayerNorm.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.output.dense.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.1.output.dense.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.output.LayerNorm.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.output.LayerNorm.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.output.dense.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.output.dense.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.key.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.key.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.query.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.query.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.value.bias
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.attention.self.value.weight
2019-08-16 00:37:47,427 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.intermediate.dense.bias
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.intermediate.dense.weight
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.output.LayerNorm.bias
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.output.LayerNorm.weight
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.output.dense.bias
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.10.output.dense.weight
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.output.LayerNorm.bias
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.output.LayerNorm.weight
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.output.dense.bias
2019-08-16 00:37:47,428 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.output.dense.weight
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.key.bias
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.key.weight
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.query.bias
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.query.weight
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.value.bias
2019-08-16 00:37:47,431 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.attention.self.value.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.intermediate.dense.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.intermediate.dense.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.output.LayerNorm.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.output.LayerNorm.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.output.dense.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.11.output.dense.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.output.LayerNorm.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.output.LayerNorm.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.output.dense.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.output.dense.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.key.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.key.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.query.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.query.weight
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.value.bias
2019-08-16 00:37:47,432 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.attention.self.value.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.intermediate.dense.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.intermediate.dense.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.output.LayerNorm.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.output.LayerNorm.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.output.dense.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.2.output.dense.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.output.LayerNorm.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.output.LayerNorm.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.output.dense.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.output.dense.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.key.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.key.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.query.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.query.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.value.bias
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.attention.self.value.weight
2019-08-16 00:37:47,433 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.intermediate.dense.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.intermediate.dense.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.output.LayerNorm.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.output.LayerNorm.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.output.dense.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.3.output.dense.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.output.LayerNorm.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.output.LayerNorm.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.output.dense.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.output.dense.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.key.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.key.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.query.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.query.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.value.bias
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.attention.self.value.weight
2019-08-16 00:37:47,434 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.intermediate.dense.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.intermediate.dense.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.output.LayerNorm.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.output.LayerNorm.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.output.dense.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.4.output.dense.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.output.LayerNorm.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.output.LayerNorm.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.output.dense.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.output.dense.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.key.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.key.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.query.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.query.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.value.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.attention.self.value.weight
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.intermediate.dense.bias
2019-08-16 00:37:47,435 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.intermediate.dense.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.output.LayerNorm.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.output.LayerNorm.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.output.dense.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.5.output.dense.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.output.LayerNorm.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.output.LayerNorm.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.output.dense.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.output.dense.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.key.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.key.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.query.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.query.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.value.bias
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.attention.self.value.weight
2019-08-16 00:37:47,436 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.intermediate.dense.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.intermediate.dense.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.output.LayerNorm.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.output.LayerNorm.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.output.dense.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.6.output.dense.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.output.LayerNorm.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.output.LayerNorm.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.output.dense.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.output.dense.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.key.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.key.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.query.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.query.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.value.bias
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.attention.self.value.weight
2019-08-16 00:37:47,437 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.intermediate.dense.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.intermediate.dense.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.output.LayerNorm.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.output.LayerNorm.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.output.dense.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.7.output.dense.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.output.LayerNorm.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.output.LayerNorm.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.output.dense.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.output.dense.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.key.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.key.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.query.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.query.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.value.bias
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.attention.self.value.weight
2019-08-16 00:37:47,438 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.intermediate.dense.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.intermediate.dense.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.output.LayerNorm.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.output.LayerNorm.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.output.dense.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.8.output.dense.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.output.LayerNorm.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.output.LayerNorm.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.output.dense.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.output.dense.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.key.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.key.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.query.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.query.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.value.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.attention.self.value.weight
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.intermediate.dense.bias
2019-08-16 00:37:47,439 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.intermediate.dense.weight
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.output.LayerNorm.bias
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.output.LayerNorm.weight
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.output.dense.bias
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.encoder.layer.9.output.dense.weight
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.pooler.dense.bias
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_bert.bert_model.pooler.dense.weight
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_token_characters._embedding._module.weight
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_token_characters._encoder._module.conv_layer_0.bias
2019-08-16 00:37:47,440 - INFO - allennlp.nn.initializers - text_field_embedder.token_embedder_token_characters._encoder._module.conv_layer_0.weight
2019-08-16 00:37:47,907 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.data.dataset_readers.dataset_reader.DatasetReader'> from params {'coding_scheme': 'BIOUL', 'tag_label': 'ner', 'token_indexers': {'bert': {'do_lowercase': 'true', 'pretrained_model': 'scibert_scivocab_uncased/vocab.txt', 'type': 'bert-pretrained', 'use_starting_offsets': True}, 'token_characters': {'min_padding_length': 3, 'type': 'characters'}}, 'type': 'conll2003'} and extras set()
2019-08-16 00:37:47,907 - INFO - allennlp.common.params - dataset_reader.type = conll2003
2019-08-16 00:37:47,907 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.data.dataset_readers.conll2003.Conll2003DatasetReader'> from params {'coding_scheme': 'BIOUL', 'tag_label': 'ner', 'token_indexers': {'bert': {'do_lowercase': 'true', 'pretrained_model': 'scibert_scivocab_uncased/vocab.txt', 'type': 'bert-pretrained', 'use_starting_offsets': True}, 'token_characters': {'min_padding_length': 3, 'type': 'characters'}}} and extras set()
2019-08-16 00:37:47,907 - INFO - allennlp.common.from_params - instantiating class allennlp.data.token_indexers.token_indexer.TokenIndexer from params {'do_lowercase': 'true', 'pretrained_model': 'scibert_scivocab_uncased/vocab.txt', 'type': 'bert-pretrained', 'use_starting_offsets': True} and extras set()
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.type = bert-pretrained
2019-08-16 00:37:47,908 - INFO - allennlp.common.from_params - instantiating class allennlp.data.token_indexers.wordpiece_indexer.PretrainedBertIndexer from params {'do_lowercase': 'true', 'pretrained_model': 'scibert_scivocab_uncased/vocab.txt', 'use_starting_offsets': True} and extras set()
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.pretrained_model = scibert_scivocab_uncased/vocab.txt
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.use_starting_offsets = True
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.do_lowercase = true
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.never_lowercase = None
2019-08-16 00:37:47,908 - INFO - allennlp.common.params - dataset_reader.token_indexers.bert.max_pieces = 512
2019-08-16 00:37:47,908 - INFO - pytorch_pretrained_bert.tokenization - loading vocabulary file scibert_scivocab_uncased/vocab.txt
2019-08-16 00:37:48,088 - INFO - allennlp.common.from_params - instantiating class allennlp.data.token_indexers.token_indexer.TokenIndexer from params {'min_padding_length': 3, 'type': 'characters'} and extras set()
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.token_indexers.token_characters.type = characters
2019-08-16 00:37:48,089 - INFO - allennlp.common.from_params - instantiating class allennlp.data.token_indexers.token_characters_indexer.TokenCharactersIndexer from params {'min_padding_length': 3} and extras set()
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.token_indexers.token_characters.namespace = token_characters
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.token_indexers.token_characters.start_tokens = None
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.token_indexers.token_characters.end_tokens = None
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.token_indexers.token_characters.min_padding_length = 3
2019-08-16 00:37:48,089 - INFO - allennlp.common.params - dataset_reader.tag_label = ner
2019-08-16 00:37:48,090 - INFO - allennlp.common.params - dataset_reader.feature_labels = ()
2019-08-16 00:37:48,090 - INFO - allennlp.common.params - dataset_reader.lazy = False
2019-08-16 00:37:48,090 - INFO - allennlp.common.params - dataset_reader.coding_scheme = BIOUL
2019-08-16 00:37:48,090 - INFO - allennlp.common.params - dataset_reader.label_namespace = labels
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/content/scibert/src/allennlp/allennlp/run.py", line 21, in <module>
run()
File "/content/scibert/src/allennlp/allennlp/run.py", line 18, in run
main(prog="allennlp")
File "/content/scibert/src/allennlp/allennlp/commands/__init__.py", line 101, in main
args.func(args)
File "/content/scibert/src/allennlp/allennlp/commands/predict.py", line 187, in _predict
predictor = _get_predictor(args)
File "/content/scibert/src/allennlp/allennlp/commands/predict.py", line 101, in _get_predictor
return Predictor.from_archive(archive, args.predictor)
File "/content/scibert/src/allennlp/allennlp/predictors/predictor.py", line 167, in from_archive
return Predictor.by_name(predictor_name)(model, dataset_reader)
File "/content/scibert/src/allennlp/allennlp/predictors/sentence_tagger.py", line 21, in __init__
self._tokenizer = SpacyWordSplitter(language=language, pos_tags=True)
File "/content/scibert/src/allennlp/allennlp/data/tokenizers/word_splitter.py", line 151, in __init__
self.spacy = get_spacy_model(language, pos_tags, parse, ner)
File "/content/scibert/src/allennlp/allennlp/common/util.py", line 290, in get_spacy_model
spacy_model = spacy.load(spacy_model_name, disable=disable)
File "/usr/local/lib/python3.6/dist-packages/spacy/__init__.py", line 21, in load
return util.load_model(name, **overrides)
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 114, in load_model
return load_model_from_package(name, **overrides)
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 135, in load_model_from_package
return cls.load(**overrides)
File "/usr/local/lib/python3.6/dist-packages/en_core_web_sm/__init__.py", line 12, in load
return load_model_from_init_py(__file__, **overrides)
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 173, in load_model_from_init_py
return load_model_from_path(data_path, meta, **overrides)
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 156, in load_model_from_path
return nlp.from_disk(model_path)
File "/usr/local/lib/python3.6/dist-packages/spacy/language.py", line 647, in from_disk
util.from_disk(path, deserializers, exclude)
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 511, in from_disk
reader(path / key)
File "/usr/local/lib/python3.6/dist-packages/spacy/language.py", line 643, in <lambda>
deserializers[name] = lambda p, proc=proc: proc.from_disk(p, vocab=False)
File "pipeline.pyx", line 643, in spacy.pipeline.Tagger.from_disk
File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 511, in from_disk
reader(path / key)
File "pipeline.pyx", line 626, in spacy.pipeline.Tagger.from_disk.load_model
File "pipeline.pyx", line 627, in spacy.pipeline.Tagger.from_disk.load_model
File "/usr/local/lib/python3.6/dist-packages/thinc/neural/_classes/model.py", line 352, in from_bytes
copy_array(dest, param[b'value'])
File "/usr/local/lib/python3.6/dist-packages/thinc/neural/util.py", line 70, in copy_array
dst[:] = src
ValueError: could not broadcast input array from shape (96) into shape (128)
2019-08-16 00:37:48,448 - INFO - allennlp.models.archival - removing temporary unarchived model dir at /tmp/tmpmyhhl9ut
dmarx commented
Yup, it was a bug related to the spacy version. Resolved by re-downloading the tokenization model via spacy.cli.download('en_core_web_sm').
ashmerj commented
@kyleclo Could you please share a sample code for NER prediction?
The link you shared is unavailable. https://github.com/allenai/allennlp/blob/master/tutorials/getting_started/predicting_paper_venues/predicting_paper_venues_pt2.md#making-predictions