marick/Midje

`expr =not=> checker` is true when `expr` evaluates to an exception

Closed this issue · 1 comments

In midje you can do

(fact (throw (ex-info "Not 1" {})) =not=> 1)

which is technically true.
That said, this can be problematic because it hides exceptions that may come up from bugs (happened to my colleague the other day)

For example:

(defn critical-code [] (throw (ex-info "important issue I want to have visibility on" {})))

;; this passes, hiding buggy behavior
(fact (critical-code) =not=> "bad")

;; produces the failure as expected
(fact (critical-code) => "good")

FAIL at (form-init5640994841513193915.clj:1)
Expected:
"good"
Actual:
clojure.lang.ExceptionInfo: important issue I want to have visibility on {}
  user$critical_code.invokeStatic(form-init5640994841513193915.clj:1)
  user$critical_code.invoke(form-init5640994841513193915.clj:1)

Proposal:
=not=> should behave the same as => when the left side throws an exception.

 (fact (critical-code) =not=> 1)
;; should produce the following result

FAIL at (form-init5640994841513193915.clj:1)
Expected:
1
Actual:
clojure.lang.ExceptionInfo: important issue I want to have visibility on {}
  user$critical_code.invokeStatic(form-init5640994841513193915.clj:1)
  user$critical_code.invoke(form-init5640994841513193915.clj:1)

This is probably not feasible, given the following example

(fact
  (throw-exception "msg") =not=> (throws "another-message" Error #(= "another-message" (.getMessage %))))

There is no way to know when the checker is validating something about the exception thrown on the right side of the arrow, so it is impossible to know when the exception should be treated as a check failure or not