weld-project/weld

"cargo test" fails with Rust 1.37.0 and LLVM/Clang 6.0.1

boegel opened this issue · 5 comments

boegel commented

I tried installing Weld 0.3.1 on top of Rust 1.37.0 and Clang 6.0.1 (both installed from source using GCC 7.3), but ran into test failures.
I ran into the same failure when using Rust 1.37.0 and Clang 8.0.1 & 9.0.0 where both were built from source using GCC 8.2.

Executed commands with output under the Details dropdown:

$ export CLANG=clang++ && cargo build --release

    Updating crates.io index
   Compiling proc-macro2 v1.0.4
   Compiling libc v0.2.62
   Compiling unicode-xid v0.2.0
   Compiling getrandom v0.1.12
   Compiling cfg-if v0.1.9
   Compiling syn v1.0.5
   Compiling memchr v2.2.1
   Compiling ppv-lite86 v0.2.5
   Compiling proc-macro2 v0.4.30
   Compiling winapi-build v0.1.1
   Compiling lazy_static v1.4.0
   Compiling unicode-xid v0.1.0
   Compiling bitflags v1.1.0
   Compiling autocfg v0.1.6
   Compiling serde v1.0.101
   Compiling ryu v1.0.0
   Compiling regex v0.2.11
   Compiling ucd-util v0.1.5
   Compiling utf8-ranges v1.0.4
   Compiling rand_core v0.4.2
   Compiling log v0.4.8
   Compiling winapi v0.2.8
   Compiling unicode-width v0.1.6
   Compiling syn v0.15.44
   Compiling semver-parser v0.7.0
   Compiling cbindgen v0.8.7
   Compiling cc v1.0.45
   Compiling remove_dir_all v0.5.2
   Compiling itoa v0.4.4
   Compiling ansi_term v0.11.0
   Compiling strsim v0.8.0
   Compiling vec_map v0.8.1
   Compiling utf8-ranges v0.1.3
   Compiling weld v0.3.1 (/tmp/weld-0.3.1/weld)
   Compiling regex-syntax v0.3.9
   Compiling bitflags v0.4.0
   Compiling encode_unicode v0.1.3
   Compiling lazy_static v0.2.11
   Compiling fnv v1.0.6
   Compiling code_builder v0.1.0
   Compiling thread_local v0.3.6
   Compiling kernel32-sys v0.2.2
   Compiling regex-syntax v0.5.6
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling textwrap v0.11.0
   Compiling c2-chacha v0.2.2
   Compiling rand_isaac v0.1.1
   Compiling rand_xorshift v0.1.1
   Compiling rand_hc v0.1.0
   Compiling rand_chacha v0.1.1
   Compiling num-traits v0.2.8
   Compiling rand_pcg v0.1.2
   Compiling rand v0.6.5
   Compiling num-integer v0.1.41
   Compiling log v0.3.9
   Compiling semver v0.9.0
   Compiling atty v0.2.13
   Compiling memchr v0.1.11
   Compiling thread-id v2.0.0
   Compiling rand_os v0.1.3
   Compiling time v0.1.42
   Compiling nix v0.5.1
   Compiling clap v2.33.0
   Compiling aho-corasick v0.5.3
   Compiling aho-corasick v0.6.10
   Compiling thread_local v0.2.7
   Compiling rand_core v0.5.1
   Compiling rand_chacha v0.2.1
   Compiling rand v0.7.2
   Compiling quote v1.0.2
   Compiling chrono v0.4.9
   Compiling quote v0.6.13
   Compiling regex v0.1.80
   Compiling uuid v0.7.4
   Compiling rustyline v1.0.0
   Compiling tempfile v3.1.0
   Compiling serde_derive v1.0.101
   Compiling llvm-sys v60.4.1
   Compiling env_logger v0.4.3
   Compiling repl v0.3.0 (/tmp/weld-0.3.1/weld-repl)
   Compiling hdrgen v0.3.0 (/tmp/weld-0.3.1/weld-hdrgen)
   Compiling serde_json v1.0.40
   Compiling toml v0.4.10
   Compiling weld-capi v0.3.0 (/tmp/weld-0.3.1/weld-capi)
    Finished release [optimized] target(s) in 1m 18s


$ cargo test --release

Output for failing test:

     Running target/release/deps/complex_loop_tests-bfb702c60ba8b3fc

running 6 tests
complex_loop_tests-bfb702c60ba8b3fc: /software/Clang/6.0.1/GCC-7.3.0-2.30/llvm-6.0.1.src/lib/IR/Constants.cpp:2055: static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *): Assertion `ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && "insertvalue indices invalid!"' failed.
error: process didn't exit successfully: `/tmp/weld-0.3.1/target/release/deps/complex_loop_tests-bfb702c60ba8b3fc` (signal: 6, SIGABRT: process abort signal)

Full output of cargo test --release:

   Compiling weld v0.3.1 (/tmp/weld-0.3.1/weld)
   Compiling hdrgen v0.3.0 (/tmp/weld-0.3.1/weld-hdrgen)
   Compiling repl v0.3.0 (/tmp/weld-0.3.1/weld-repl)
   Compiling weld-capi v0.3.0 (/tmp/weld-0.3.1/weld-capi)
    Finished release [optimized] target(s) in 19.41s
     Running target/release/deps/hdrgen-8c616f06db4a3dd8

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/repl-f2e12a0e7733801e

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/weld-6bc0ffa5b90761a4

running 75 tests
test ast::constructors::makevector_empty_test ... ok
test ast::constructors::binop_test ... ok
test ast::constructors::comparison_test ... ok
test ast::constructors::builder_exprs_test ... ok
test ast::constructors::literal_test ... ok
test data::size_check ... ok
test ast::hash::test_compare_different_symbols_ne ... ok
test ast::hash::test_compare_different_symbols ... ok
test ast::hash::test_compare_same ... ok
test ast::hash::test_lambda ... ok
test optimizer::transforms::cse::builder_test ... ok
test ast::type_inference::infer_types_test ... ok
test optimizer::transforms::cse::builder_test_3 ... ok
test ast::cmp::compare_expressions ... ok
test optimizer::transforms::cse::builder_test_4 ... ok
test ast::uniquify::parse_and_print_uniquified_expressions ... ok
test optimizer::transforms::cse::basic_test ... ok
test optimizer::transforms::cse::builder_test_2 ... ok
test optimizer::transforms::cse::alias_test ... ok
test optimizer::transforms::cse::for_test ... ok
test optimizer::transforms::cse::for_test_2 ... ok
test optimizer::transforms::cse::if_test ... ok
test optimizer::transforms::cse::if_test_3 ... ok
test optimizer::transforms::cse::if_test_2 ... ok
test optimizer::transforms::cse::if_test_4 ... ok
test optimizer::transforms::cse::let_test ... ok
test optimizer::transforms::cse::for_test_4 ... ok
test optimizer::transforms::cse::for_test_3 ... ok
test optimizer::transforms::cse::if_test_5 ... ok
test optimizer::transforms::short_circuit::compound_or ... ok
test optimizer::transforms::cse::if_test_6 ... ok
test optimizer::transforms::short_circuit::simple_or ... ok
test optimizer::transforms::short_circuit::compound_and ... ok
test optimizer::transforms::cse::if_test_7 ... ok
test optimizer::transforms::cse::if_test_8 ... ok
test optimizer::transforms::unroller::large_merger_loop ... ok
test optimizer::transforms::cse::nesting_test ... ok
test optimizer::transforms::cse::many_subexprs_test ... ok
test optimizer::transforms::short_circuit::complex_and_or ... ok
test optimizer::transforms::short_circuit::simple_and ... ok
test optimizer::transforms::short_circuit::predicated_if ... ok
test optimizer::transforms::unroller::simple_appender_loop ... ok
test optimizer::transforms::unroller::zipped_appender_loop ... ok
test optimizer::transforms::vectorizer::predicated_appender ... ok
test optimizer::transforms::vectorizer::non_vectorizable_type ... ok
test optimizer::transforms::vectorizer::unpredicated_merger ... ok
test optimizer::transforms::vectorizer::non_vectorizable_expr ... ok
test optimizer::transforms::vectorizer::simple_merger ... ok
test optimizer::transforms::vectorizer::simple_appender ... ok
test optimizer::transforms::vectorizer::predicated_merger ... ok
test optimizer::transforms::unroller::zipped_merger_loop ... ok
test syntax::macro_processor::invalid_type_alias ... ok
test syntax::macro_processor::redefinition_with_type_alias_in_type_alias ... ok
test syntax::macro_processor::expr_with_type_alias ... ok
test syntax::macro_processor::nested_type_alias ... ok
test optimizer::transforms::inliner::inline_lets ... ok
test syntax::macro_processor::invalid_type_alias_in_type_alias ... ok
test syntax::macro_processor::type_alias_in_type_alias ... ok
test syntax::macro_processor::basic_type_alias ... ok
test syntax::macro_processor::overwrite_type_alias ... ok
test syntax::macro_processor::basic_macros ... ok
test syntax::parser::parse_and_print_for_expressions ... ok
test syntax::parser::read_to_end_of_input ... ok
test syntax::parser::parse_and_print_simple_expressions ... ok
test syntax::parser::parse_and_print_literal_expressions ... ok
test syntax::macro_processor::macros_introducing_symbols ... ok
test syntax::parser::basic_parsing ... ok
test optimizer::transforms::loop_fusion::simple_vertical_loop_fusion ... ok
test optimizer::transforms::unroller::simple_merger_loop ... ok
test optimizer::transforms::short_circuit::complex_and_or_2 ... ok
test syntax::parser::operator_precedence ... ok
test optimizer::transforms::vectorizer::zipped_input ... ok
test optimizer::transforms::loop_fusion::simple_horizontal_loop_fusion ... ok
test syntax::macro_processor::standard_macros ... ok
test syntax::tokenizer::basic_tokenize ... ok

test result: ok. 75 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/appender_tests-ccc27335d9d7c215

running 5 tests
test simple_parallel_for_appender_loop ... ok
test simple_for_appender_loop ... ok
test large_unaryop_for_appender_loop ... ok
test simple_parallel_for_multi_appender_loop ... ok
test complex_parallel_for_appender_loop ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/basic_tests-d76fcc47b63a4cd6

running 25 tests
test iterate_non_parallel ... ok
test bool_eq ... ok
test i32_cast ... ok
test iterate_with_parallel_body ... ok
test filter_length ... ok
test f64_cast ... ok
test flat_map_length ... ok
test let_statement ... ok
test maxmin ... ok
test basic_program ... ok
test map_zip_loop ... ok
test if_statement ... ok
test if_for_loop ... ok
test program_with_args ... ok
test multiple_casts ... ok
test comparison ... ok
test negated_arithmetic ... ok
test simple_length ... ok
test struct_vector_literals ... ok
test negation_double ... ok
test negation ... ok
test reused_variable ... ok
test not ... ok
test serial_parlib_test ... ok
test float_literals ... ok

test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/binop_tests-34571d78fb7705e6

running 5 tests
test struct_cmp ... ok
test simple_binop ... ok
test string_cmp ... ok
test vector_cmp ... ok
test substring_cmp ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/release/deps/complex_loop_tests-bfb702c60ba8b3fc

running 6 tests
complex_loop_tests-bfb702c60ba8b3fc: /software/Clang/6.0.1/GCC-7.3.0-2.30/llvm-6.0.1.src/lib/IR/Constants.cpp:2055: static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *): Assertion `ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && "insertvalue indices invalid!"' failed.
error: process didn't exit successfully: `/tmp/weld-0.3.1/target/release/deps/complex_loop_tests-bfb702c60ba8b3fc` (signal: 6, SIGABRT: process abort signal)

Is this a known issue, or am I overlooking something?

Side notes:

  • the README mentions that LLVM is required, but you actually need Clang, since clang++ is used during the build;
  • the README also mentions "To build Weld, you need ... and LLVM 6.0 or newer". This lead me to believe that LLVM 7/8/9 would also work (but #468 says 6.0.x is required).
sppalkia commented

Hmm, I'm not able to reproduce this, what do you see when you run llvm-config --version?

Good points re: README, I will fix those.

boegel commented

As expected, 6.0.1 (part of the Clang installation)

$ llvm-config --version
6.0.1
$ which llvm-config
/software/Clang/6.0.1-GCC-7.3.0-2.30/bin/llvm-config
sppalkia commented
zao commented

Clang/LLVM built with this build system typically has -DLLVM_ENABLE_ASSERTIONS=ON.

I've tested four different builds of the toolchain:

  • Ubuntu's stock LLVM 6.0.0-1ubuntu2,
  • three versions built from source:
    • 6.0.0 with assertions,
    • 6.0.1 with assertions,
    • 6.0.1 without assertions.

The distro's compiler and the 6.0.1 without assertions pass the test suite, the two builds with assertions bail out in the same manner that @boegel's did.

sppalkia commented

Okay, I built it from source and was able to reproduce this. Will post a patch shortly.