rust-lang/rustc-dev-guide

Guide answers "what" but not "why" and "how" for codegen tests

workingjubilee opened this issue · 0 comments

The section on codegen tests is bereft of an explanation of how to use FileCheck, and I do not think the documentation for FileCheck is the best example for it. The documentation for FileCheck focuses on its usage as a CLI tool first, but we are not passing random args to FileCheck!

In fact, it's probably exactly the reverse order for what we want. What is probably most important is how and when to use the regex capturing, introduced at the end, since most tests shouldn't break if a variable changes its name but uses a few different ones: we don't want to focus on LLVM's exact behavior like that. Then we usually want people to use CHECK-LABELs, again, introduced just before regex usage. Then we want to discuss when to use CHECK-NEXT, CHECK-NOT, and CHECK-DAG, and when not to use them.

In our actual usage, CHECK, the directive they introduce first and focus on using, is mostly a directive for corraling CHECK-NEXT and CHECK-DAGs when CHECK-LABEL does not suffice for that, as CHECK is itself the hardest-to-contain directive.

A step-by-step explanation for

  • how to write an okay FileCheck test
  • how to write a really good one

would be better.

And neither of these answer why we use them, which is particular to the Rust codebase. We can't simply upstream all our tests unless LLVM adds rustc to the test suite, as we want e.g. our MIR opts to be taken into account when running our codegen tests.