icppWorld/icpp_llm

110M TinyStories hits instruction limit for non-empty prompt

icppWorld opened this issue · 0 comments

For the 110M TinyStories canister, this works fine:

dfx canister call --network ic  "obk3p-xiaaa-aaaag-ab2oa-cai" new_chat '()'
dfx canister call --network ic  "obk3p-xiaaa-aaaag-ab2oa-cai" inference '(record {prompt = "" : text; steps = 10 : nat64; temperature = 0.0 : float32; topp = 0.9 : float32; rng_seed = 0 : nat64;})'

But this fails:

dfx canister call --network ic  "obk3p-xiaaa-aaaag-ab2oa-cai" new_chat '()'
dfx canister call --network ic  "obk3p-xiaaa-aaaag-ab2oa-cai" inference '(record {prompt = "Bobby" : text; steps = 10 : nat64; temperature = 0.0 : float32; topp = 0.9 : float32; rng_seed = 0 : nat64;})'

Error: Failed update call.
Caused by: Failed update call.
  The replica returned a replica error: Replica Error: reject code CanisterError, reject message Canister obk3p-xiaaa-aaaag-ab2oa-cai exceeded the instruction limit for single message execution., error code None