Non-zero offsets to null pointers leads to UBSAN/ASAN errors
Closed this issue · 8 comments
Describe the bug
The use of BYTE pointers to calculate offsets in the byte stream (such as in this code) leads to errors when ZSTD is compiled with clang 10 and UBSAN/ASAN sanitizer options.
In cases where the BYTE pointer equals a null pointer, the sanitizer checks fail (the reason is explained here).
A possible fix is to use (uintptr_t) instead of BYTE* to calculate the offsets.
To Reproduce
These errors were generated on UBSAN/ASAN tests run by the CRAN infrastructure (in this case on my R package fst). The exact compiler options used for those runs are documented here. The errors can be reproduced by using compiler options -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-sanitize=alignment -fno-omit-frame-pointer -frtti.
Additional context
The errors were found on ZSTD release v1.4.4. For completeness, I've pasted the errors below:
ZSTD/compress/zstd_compress.c:1323:46:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_compress_internal.h:876:41:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_compress_internal.h:877:33:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_double_fast.c:99:62:
runtime error: applying zero offset to null pointer
ZSTD/compress/zstd_opt.c:420:42:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_opt.c:568:42:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_opt.c:612:55:
runtime error: applying non-zero offset 4294967294 to null pointer
ZSTD/compress/zstd_lazy.c:663:61:
runtime error: applying zero offset to null pointer
ZSTD/compress/zstd_lazy.c:495:42:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_compress.c:1323:46:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_compress_internal.h:876:41:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_compress_internal.h:877:33:
runtime error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_opt.c:420:42: runtime
error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_opt.c:568:42: runtime
error: applying non-zero offset 1 to null pointer
ZSTD/compress/zstd_opt.c:612:55: runtime
error: applying non-zero offset 8445 to null pointer
thanks for considering my issue and providing a fantastic library!
uintptr_t is, unfortunately, not portable.
We try to avoid its usage as we maintain c90 compatibility for this code base.
That being said, I'm sure there are other possible solutions.
thanx @Cyan4973, I didn't know that :-). I used uintptr_t to silence those errors but unfortunately that's not an acceptable fix then.
Any updates on this issue? Patching a no_sanitize("undefined") is possible but it does not fix the undefined behavior caused by the library
We should no longer have any non-zero offsets added to NULL pointers, so I'm closing this issue. Please re-open if you see any more.
@terrelln - Since there isn't a commit currently associated with this issue, it is difficult to know what the first release of zstd containing the fix might be. Could you please update this issue with that information? I'd also be interested to know the exact commit because I may want to cherry-pick the fixes, if appropriate/reasonable. Thanks!
I believe this objective was gradually achieved across multiple commits, so there's not just one place to look at.
@Cyan4973 - Thanks. Do you happen to know what the first release that contains all those commits might be?
I would expect v1.5.0 to contain most if not all of them.