TypeError: 'QuantoConfig' object is not subscriptable

[mehulgupta2016154]

While running the example code in Readme.md
`from local_gemma import LocalGemma2ForCausalLM
from transformers import AutoTokenizer
import os

os.environ['HUGGINGFACEHUB_API_TOKEN'] = ''
os.environ['HF_TOKEN'] = ''

model = LocalGemma2ForCausalLM.from_pretrained("google/gemma-2-9b",token='')
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b")

model_inputs = tokenizer("The cat sat on the mat", return_attention_mask=True, return_tensors="pt")
generated_ids = model.generate(**model_inputs.to(model.device))

decoded_text = tokenizer.batch_decode(generated_ids)`

Kindly help

Note: I've installed quanto explicitly when prompted in a previous error

[julien-c]

please rotate the tokens above^

[mehulgupta2016154]

Thanks for notifying. Any idea why I'm getting the error?

[sanchit-gandhi]

Hey @mehulgupta2016154 - thanks for reporting! Note that you can save your HF token locally to your cache by running the following from a terminal shell:

huggingface-cli login

This will save you having to pass the token argument to .from_pretrained.

Could you please paste the full traceback you get when running this cell? And also the output of transformers-cli env? Thanks