bats-core/bats-assert

[Feature] Option to explain the result

jbriales opened this issue · 2 comments

Optional explanation on assertion result

Sometimes we write checks based not directly in the output but in some derived result.

E.g. consider the case of checking for the nr of output lines to be 1 as assert_equal "$(wc -l <<<"$output")" 1. If this fails, we'd want to show the full output on error to understand better the root-cause for the failure.

This could be done by e.g. having an extra option --error-explanation with the custom message to show on error. This is partially inspired by the result_listener feature in GTest matchers to explain the match result.

Hence instead of

  `assert_equal "$(wc -l <<<"$output")" 1' failed

-- values do not equal --
expected : 1
actual   : 3
--

we could do

  `assert_equal "$(wc -l <<<"$output")" 1 --error-explanation="there should be 1 line instead of\n$output"' failed

-- values do not equal --
expected : 1
actual   : 3
explanation : there should be 1 line instead of
foo
bar
quz
--

Is this a feature that could be of general interest?

I agree this would be useful. For example, I am checking 3 nodes to ensure that there is one Kubernetes API server per node. The error message as it stands is not intuitive, and I would love to be able to provide an explanation for my coworkers.

 ✗ Check if only one kube-apiserver container per node [224]
`assert_line --index 0 --partial kubernetes03' failed