byT5 with version 0.1.2

[kimgerdes]

hi there,
it seems that the newest version of simpleT5 does no longer work with byT5. The line elif model_type == "byt5": is commented out. The newest version of transformers seems to use a new type of tokenizer T5TokenizerFast and ByT5TokenizerFast does not exist. Any ideas about how to fix that?

[Shivanandroy]

In the latest version of simpleT5 (0.1.2), I have dropped support for ByT5 models for now, because of transformers and ONNX version conflict. This will be fixed in upcoming releases

For now, you can train byt5 models with simplet5==0.1.1.

# pip install simplet5==0.1.1
from simplet5 import SimpleT5

# instantiate
model = SimpleT5()

# load (supports t5, mt5, byt5)
model.from_pretrained("byt5","google/byt5-small")

# train
model.train(train_df=train_df,
            eval_df=eval_df, 
            source_max_token_len = 128, 
            target_max_token_len = 128,
            batch_size = 8,
            max_epochs = 5,
            use_gpu = True,
            outputdir = "outputs",
            early_stopping_patience_epochs = 0,
            precision = 32
            )

# load trained T5 model
model.load_model("byt5","path/to/trained/model/directory", use_gpu=False)

# predict
model.predict("input text for prediction")

[kimgerdes]

Ok, that works. Thanks a lot for the quick answer!
What is actually the advantage of moving to 0.1.2? I've failed to find a changelog.
thanks again