modal-labs/modal-examples

how to run llama3.2 1b model ?

Closed this issue · 1 comments

Arslan-Mehmood1 commented

using this

06_gpu_and_ml/llm-serving/text_generation_inference.py

I'm unable to get generated text correctly.
image

charlesfrye commented

Hey there!

We reserve the GitHub Issues page for issues with the examples as they are written, but this looks like an issue with the example after it has been edited.

We provide support for general use of the Modal platform in our Community Slack, https://modal.com/slack.

The short answer is that it looks like the chat template here is wrong.