BurntSushi/utf8-ranges

Test suite executes UB by attempting to construct invalid chars

saethlin opened this issue · 1 comments

I see this crate is deprecated but since deprecating it you've published at least one version of fst which depends on it, and the latest commit in the fst repo still depends on this crate. So I guess I'm a bit confused by the deprecated status, but I'd also understand if you just close this as a non-issue 🤷

This code in the test suite is unconditional UB because it attempts to construct an invalid char. Miri immediately flags this (which is how I ran into this).

utf8-ranges/src/lib.rs

Lines 463 to 464 in 8d38a93

for cp in 0xD800..0xE000 {
let c = unsafe { ::std::mem::transmute(cp) };

But you should also be aware that the standard library is veeery slooowly adding debug assertions for preconditions which are detectable at runtime. So these tests are unlikely to break any time soon, but due to this UB these tests could start failing at any time.

Aye, thanks for the report! I've fixed this (and removed the other use of unsafe in this crate) in utf8-ranges 1.0.5. I'm not sure why I wrote this code. According to the commit log, I wrote it before from_u32_unchecked was stabilized in Rust 1.5. It's plausible that I just wasn't as aware or astute when it came to type validity invariants back then. When I moved this code into regex-syntax, I did indeed fix this bug, but I never ported that fix back into this crate. (Which is what I've just done.)

Also, yes, this crate is deprecated. When I deprecated it, I had forgotten that it was still used in fst. I do hope to fix that some day. In practice though, I'll likely keep this crate working. Although, I now see that its CI is busted.