Use of TPM2_HR_PERSISTENT triggers undefined behaviour

[polarina]

Take for example this program:

#include <tss2/tss2_tpm2_types.h>

int main() {
	if (TPM2_HR_PERSISTENT) {
		return 0;
	}

	return 1;
}

Compile:

gcc -fsanitize=undefined ubsan.c

Execute:

$ ./a.out 
ubsan.c:4:6: runtime error: left shift of 129 by 24 places cannot be represented in type 'int'

[AndreasFuchsTPM]

I guess we are casting wrongly.

Currently we have

#define TPM2_HR_PERSISTENT        ((TPM2_HC) (TPM2_HT_PERSISTENT << TPM2_HR_SHIFT))

I guess this should be

#define TPM2_HR_PERSISTENT        (((TPM2_HC) TPM2_HT_PERSISTENT) << TPM2_HR_SHIFT)

Would you agree ?

[polarina]

Would you agree ?

Yes, that would fix the problem. Both arguments to << would then be of type uint32_t which vibes with the C integer promotion rules.