Issues on evaluating latency using TVM.

[rkdgmlqja]

Hi I’m currently working on compiling I-ViT using TVM. On this project, The error appears.

Check failed: value < 1LL << (dtype.bits() - 1) (192 vs. 128) : ValueError: Literal value 192 exceeds maximum of int8

by changing value 192 lower than 128 on build_model.py seems to sove the issue.
if name == 'deit_tiny_patch16_224': #embed_dim = 192 embed_dim = 92 num_heads = 3
But, strictly speaking, 'But this method' involves arbitrarily modifying the model's structure, so it is not an appropriate solution.
should changing TVM's version solve this issue?

Thanks always.

[gihwan-kim]

@zkkli , @rkdgmlqja
I have a same issue.
if embed_dim is higher then 128(int8 range), ValueError occurs. But in buid_model.py file, every embed_dim is higher then 128. So it occurs error inevitably.
How to solve this problem?

[rkdgmlqja]

The Error was caused by quantized layer norm from layers.py

def quantized_layernorm(data, 
                    bias_int):
    data = relay.cast(data, 'int32')
    mean = relay.mean(data, axis=2, keepdims=True)
    data = data - mean
    data_sq = data * data

This should fix it

[dedoogong]

hi, @rkdgmlqja . did you successfully run tvm deployment(evaluate_accuracy.py, evaluate_latency.py)?
I got stuck with it.
I've tried it in the same environment as the author guided(tvm 0.9dev0 from source build, timm 0.4.2).
but the inference results(top 5) was totally different from torch.
also, I always failed in the auto tuning progress with a unknown sudden error.
what version of tvm, timm or some other settings did you use? is there some more things I should change to run it correctly?
if possible please help me~!