Question: calculation of memory length for validation in TransformerXL
Opened this issue · 0 comments
whatever60 commented
Hi, very helpful repo, learned a lot from it.
I got a question about an implementation detail in TransformerXL.
In the transformer_xl_from_scratch notebook, the memory length during validation is calculated as val_memory_length + train_bptt - val_bptt.
Why aren't it just set to val_memory_length?
Looking forward to reply.