amasky/ram

why np.identity?

machanic opened this issue · 1 comments

in train.py file:

if not args.lstm:
    data = model.core_hh.W.data
    data[:] = np.identity(data.shape[0], dtype=np.float32)

Actually, this is not the way in the original paper. According to arXiv:1504.00941, initializing the weight of RNN with identity matrix may improve performance. But I think this does not affect in this case. I will delete this code later.