batch_first should be True when initializing the LSTM?

Here:

Line 34 in 67b690b

self.rnn = nn.LSTM(300, 300, num_layers=2, bidirectional=True)

the LSTM is defined with batch_first=False (default value).
However, when you pass the inputs to the LSTM and get outputs from the LSTM, you use batch_first=True.

cove/cove/encoder.py

Lines 51 to 52 in 67b690b

    
           outputs, hidden_t = self.rnn(pack(inputs[indices], lens.tolist(), batch_first=True), hidden) 
        
           outputs = unpack(outputs, batch_first=True)[0]

Will this cause inconsistency and generate wrong CoVe vectors?

Hmm, I'm not sure exactly how PyTorch handles this, but I'll change the LSTM to also have batch_first=True to be safe.

3d57c2f Should take care of that.

	outputs, hidden_t = self.rnn(pack(inputs[indices], lens.tolist(), batch_first=True), hidden)
	outputs = unpack(outputs, batch_first=True)[0]