eos token does not work in batch mode generation

Question

eos token does not work in batch mode generation

Closed this issue 3 years ago · 4 comments

When generating the sequence with current code it seems the eos_token will work when generating one sequence at a time https://github.com/lucidrains/h-transformer-1d/blob/main/h_transformer_1d/autoregressive_wrapper.py#L59

Answer 1 · 2021-08-08T16:09:06.000Z

ohh, this is indeed a bug, and i'll fix this today

Answer 2 · 2021-08-08T16:47:27.000Z

@tmphex let me know if this e3cb1af works for you

Answer 3 · 2021-08-10T04:24:26.000Z

Thanks @lucidrains for fixing it quickly. I think its the same issue in x-transformers repo too. https://github.com/lucidrains/x-transformers/blob/main/x_transformers/autoregressive_wrapper.py#L80

Answer 4 · 2021-08-10T21:45:32.000Z

Thanks @lucidrains for fixing it quickly. I think its the same issue in x-transformers repo too. https://github.com/lucidrains/x-transformers/blob/main/x_transformers/autoregressive_wrapper.py#L80

haha yup, i'll fix it there too