Create a rules test definition macro that can be shared between Clojure and ClojureScript

[WilliamParker]

Our tests for the ClojureScript version of Clara are far less extensive than our tests for the Clojure version. A major reason for this is that testing is easier in Clojure than in ClojureScript since Clojure supports on-the-fly creation of rules sessions, while the ClojureScript requires compile-time creation of rules sessions. This prevents us from writing tests like this for ClojureScript:

(deftest test-simple-query
  (let [cold-query (dsl/parse-query [] [[Temperature (< temperature 20) (= ?t temperature)]])

        session (-> (mk-session [cold-query])
                    (insert (->Temperature 15 "MCI"))
                    (insert (->Temperature 10 "MCI"))
                    (insert (->Temperature 80 "MCI"))
                    fire-rules)]

    (is (= #{{:?t 15} {:?t 10}}
           (set (query session cold-query))))))

The alternative in ClojureScript of creating a ClojureScript namespace or creating a Clojure namespace containing the rules like test-rules-data is inconvenient and, multiplied over the large number of test cases we have, reduces readability of the tests considerably.

The end result is that most of our tests only run for Clojure. While this works to a large degree since most of the codebase is shared between Clojure and ClojureScript, this is not always the case. In particular, a variety of performance optimizations we've made for the Clojure version rely on JVM interop, which obviously won't work in ClojureScript. In particular, the memory uses Java data structures extensively. The holes this creates in our testing of the ClojureScript version are problematic; there are any number of ways in which we could cause a regression in the ClojureScript version without impacting the Clojure version.

Ideally we'd have a common testing codebase between Clojure and ClojureScript that kept the desirable characteristics of our current Clojure tests. I think this can be achieved with a custom test definition macro that would look very similar to our current tests but would create the sessions in ClojureScript at compile time and at runtime in Clojure. I'm envisioning something like

(def-rules-test simple-test
  :rules [cold-rule (....)
          hot-rule (....)]
  :queries [cold-query]
  :sessions [session1 [cold-rule hot-rule] {opts-here}
             session2 [cold-rule] {opts-here}]
  (body-of-tests-here))

Behind the scenes, this macro would call parse-rule on each of the unevaluated forms for rules, in this case cold-rule and hot-rule. For each session binding created, the rules would be passed to logic like that in defsession prior to the unevaluated code that it returns. The generated test would ultimately look something like this after macroexpansion:

(deftest simple-test
   (let [session1 (assemble-session ....)
         session2 (assemble-session)
         cold-query (...)])
  test-body-here)

With some refactoring of the logic in defsession I believe most of the logic for actually creating the session could be shared between this macro and the real code. Queries would need to be let-bound for calls to query to work, but I don't see similar restrictions for rules. Queries and rules could be in the same part of the test or in different parts; I'm not sure which is the best approach.

Note that this would not restrict the code that could be present in the body; it would be arbitrary code and could create additional let bindings, have many different test cases, etc. This code in the body would be responsible for creating facts, inserting them, firing the rules, and so forth. My thinking is that we'd want to restrict the scope of the macro to only things that need to be done at compile time. I'd prefer to avoid having an overly restrictive test macro that can't handle additional cases that may come up or that could require lots of boilerplate code.

[rbrush]

Great idea. I had struggled with how to make more of our tests portable, and this is better than anything I had come up with.

[mrrodriguez]

I like this. I think it'd be a huge win to have all the tests run for both clj and cljs too. It'd definitely help keep them aligned.

[WilliamParker]

This was fixed by commit c67b41b in PR #301

We still need to figure out why the Travis build doesn't pick up our common test namespaces properly before making broad use of this macro; I have logged #302 for that investigation. However, everything within the scope of this issue is complete, so I'm going to close this issue.