word embeddings should be fixed during training?

In your paper Learned in Translation: Contextualized Word Vectors (McCann et. al. 2017), it says

We used the CommonCrawl-840B GloVe model for English word vectors, which were completely fixed during training, so that the MT-LSTM had to learn how to use the pretrained vectors for translation.

However, in your code

cove/cove/encoder.py

Lines 29 to 33 in 67b690b

    
           if n_vocab is not None: 
        
               self.embed = True 
        
               self.vectors = nn.Embedding(n_vocab, 300) 
        
               if vectors is not None: 
        
                   self.vectors.weight.data = vectors

the Glove embeddings are not fixed.

Right, in my paper, I found that the weights should be fixed for better performance. I just wanted to leave this a little more flexible in case others wanted to try fine-tuning on different tasks.

	if n_vocab is not None:
	self.embed = True
	self.vectors = nn.Embedding(n_vocab, 300)
	if vectors is not None:
	self.vectors.weight.data = vectors