replace for-loop in forward decoder

[Tangzy7]

Hello I also found that forward decoder (for-loop) is time-consuming, do you have a good solution instead of for-loop?

[howardyclo]

@Tangzy7 Hello, I study the codebase of OpenNMT-py, and found that if the decoder without input feed can be implemented without for loop. Please refer to: https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/Models.py#L388

But if using the decoder with input feed, I think it must be implemented with for-loop:
https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/Models.py#L490

In inference mode, a for-loop is required since we don't know the ground-truth targets (model predict token by its previous token).