Colab notebook has error, numpy array used instead of torch

[microcoder-py]

while True:
    user_message = input("\nYour message: ")
    messages.append(dict(
        role="user",
        content=user_message
    ))
 

    out = model.generate(input_ids=input_ids, max_length=2000, temperature=0.9, top_p=0.7, eos_token_id=tokenizer.eos_token_id)

    decoded = tokenizer.batch_decode(out)
    messages.append(dict(
        role="assistant",
        content=decoded[0].split("<|assistant|>\n")[-1])
    )

    print("Model:", decoded[0].split("<|assistant|>\n")[-1])

Error:

AttributeError                            Traceback (most recent call last)
[<ipython-input-6-8618df48d4c0>](https://localhost:8080/#) in <cell line: 2>()
      9     input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
     10 
---> 11     out = model.generate(input_ids=input_ids, max_length=2000, temperature=0.9, top_p=0.7, eos_token_id=tokenizer.eos_token_id)
     12 
     13     decoded = tokenizer.batch_decode(out)

18 frames
[/usr/local/lib/python3.10/dist-packages/causal_conv1d/causal_conv1d_interface.py](https://localhost:8080/#) in forward(ctx, x, weight, bias, seq_idx, activation)
     17             x = x.contiguous()
     18         bias = bias.contiguous() if bias is not None else None
---> 19         seq_idx = seq_idx.contiguous() if seq_idx is not None else None
     20         ctx.save_for_backward(x, weight, bias, seq_idx)
     21         ctx.activation = activation in ["silu", "swish"]

AttributeError: 'str' object has no attribute 'contiguous'

Line most likely causing error, torch conversion did not happen, numpy arrays do not have any contigous method. Fix this to explicitly cast as torch tensor:

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")

[LuciAkirami]

So like what is the fix now? The input_ids is indeed of type tensors

[justusmattern27]

@microcoder-py Have you modified the code in the notebook in some way? The return_tensors='pt' argument is already present in the code if I'm not mistaken

[LuciAkirami]

The code is run as it is from the colab @justusmattern27 and still it gives the above error. Checked the type of input_ids before feeding them to output and they are indeed of type tensor but still getting the above error

[justusmattern27]

@LuciAkirami got it, thanks - I ran into it myself now, something that should be a tensor seems to be a string here, which is pretty weird. The notebook worked a few days ago, so I guess it could be due to updates in the dependencies. Will look into it later

[justusmattern27]

It was indeed due to wrong versions of the dependencies, I pinned them now.
@LuciAkirami @microcoder-py This should be fixed now, feel free to let me know if you run into the error again. Otherwise, I'll close the issue soon :)

[LuciAkirami]

Cooool @justusmattern27 , now it's working

[justusmattern27]

nice!