Stack overflow in AVX2 code
thomwiggers opened this issue · 1 comments
thomwiggers commented
Compiling test_kyber512
with -fsanitize=address,undefined
on gcc 10.1.0
leads to the following:
=================================================================
==288165==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55b880d170b8 at pc 0x55b880c9b0fc bp 0x7fff1a3c8950 sp 0x7fff1a3c8940
READ of size 16 at 0x55b880d170b8 thread T0
#0 0x55b880c9b0fb in pqcrystals_kyber512_avx2_rej_uniform_avx (/home/thom/git/phd/kyber/avx2/test_kyber512+0x710fb)
#1 0x55b880c897c6 in pqcrystals_kyber512_avx2_gen_matrix (/home/thom/git/phd/kyber/avx2/test_kyber512+0x5f7c6)
#2 0x55b880c8b5e0 in pqcrystals_kyber512_avx2_indcpa_keypair (/home/thom/git/phd/kyber/avx2/test_kyber512+0x615e0)
#3 0x55b880c88a61 in pqcrystals_kyber512_avx2_keypair (/home/thom/git/phd/kyber/avx2/test_kyber512+0x5ea61)
#4 0x55b880d13b9a in test_keys.isra.0 (/home/thom/git/phd/kyber/avx2/test_kyber512+0xe9b9a)
#5 0x55b880c888ba in main (/home/thom/git/phd/kyber/avx2/test_kyber512+0x5e8ba)
#6 0x7f366960a001 in __libc_start_main (/usr/lib/libc.so.6+0x27001)
#7 0x55b880c8896d in _start (/home/thom/git/phd/kyber/avx2/test_kyber512+0x5e96d)
0x55b880d170c0 is located 0 bytes to the right of global variable 'idx' defined in 'rejsample.c:8:22' (0x55b880d168c0) of size 2048
SUMMARY: AddressSanitizer: global-buffer-overflow (/home/thom/git/phd/kyber/avx2/test_kyber512+0x710fb) in pqcrystals_kyber512_avx2_rej_uniform_avx
Shadow bytes around the buggy address:
0x0ab79019adc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ab79019add0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ab79019ade0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ab79019adf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ab79019ae00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0ab79019ae10: 00 00 00 00 00 00 00[00]f9 f9 f9 f9 00 00 00 00
0x0ab79019ae20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ab79019ae30: 00 00 00 04 f9 f9 f9 f9 06 f9 f9 f9 f9 f9 f9 f9
0x0ab79019ae40: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 f9 f9
0x0ab79019ae50: f9 f9 f9 f9 00 00 00 00 00 00 00 04 f9 f9 f9 f9
0x0ab79019ae60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==288165==ABORTING
gregorseiler commented
Thanks! The problem was using _mm_loadu_si128 instead of _mm_loadl_epi64 to read 8 bytes from the idx table. It is fixed now.