RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding).

[foton263]

I am trying your excellent and interesting code and thank you for sharing it.
Since running CUDA in WSL Ubundu is somewhat hard to setup, I tried to for starters to run the demo_paraphrase.py in a conda environment under Windows 10 and this gave me the following error:

RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding).
I guess there is some discrepancy between Windows and Linux, in the way the transformers library behaves.
To make it short, I fixed it by adding the following lines to your utils.py in get_logits function

def get_logits(model, iteration, generated, segments, style_content_vectors, past):
# fixing LongTensor bug in windows systems #########################
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
generated = generated.type(torch.LongTensor)
segments = segments.type(torch.LongTensor)
generated = generated.to(device)
segments = segments.to(device)
###########################################################

I think this kind of solution is generic. I worder if this code works now under Linux.

[martiansideofthemoon]

Thanks for sharing! Could you submit this as a pull request? I could then confirm it works in Linux and merge it to the codebase.