Provide example for testing with junit xml output

[peterneubauer]

Hi there,
I'm trying to execute the equivalent of

riemann test riemann.config.clj

where I have something like

 (tests

 (defn add-test
   "Add a test to the given namespace. The body of the test is given as
   the thunk test-fn. Useful for adding dynamically generated deftests."
   [name ns test-fn & [metadata]]
   (intern ns (with-meta (symbol name) (merge metadata {:test #(test-fn)})) (fn []))
   )

 (defmethod clojure.test/report :begin-test-var
   [m]
   (println "\u001B[32mTesting" (-> m :var meta :name) "\u001B[0m"))


 (defn add_threshold_test [test_name, target_tap_count, events]
   (println "events" events)
   (add-test (symbol test_name), (symbol "riemann.config-test"),
             (fn []
               (is
                (=
                 (count
                  (:thresholds
                    (inject! events)))
                 target_tap_count)))))

 (doseq [test_data
         [["test1"
           2
           [{:service "xxx.count"
             :metric  2000
             :host    ""}
            {:service "xxx.count"
             :host    ""
             :metric  500}]]]
   (add_threshold_test
    (get test_data 0)
    (get test_data 1)
    (get test_data 2))))

This works good and gives me terminal output.

With leiningen and the junit plugin in order to get XML output I configure my project like this:

(defproject example "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [riemann/riemann "0.2.14"]]
  :plugins               [[lein-test-report-junit-xml "0.2.0"]]
  :test-report-junit-xml {:output-dir "tests/test_results"}

  )
;; never do this - HTTP repos allow
(require
 'cemerick.pomegranate.aether)

(cemerick.pomegranate.aether/register-wagon-factory!
 "http" #(org.apache.maven.wagon.providers.http.HttpWagon.))

(ns leiningen.rtest
  (:use [leiningen.core.eval :only [eval-in eval-in-project]]))

(defn rtest [project test-file]
  (eval-in
   project
   `(do
     (require
      'riemann.bin)
     (riemann.bin/-main "test" ~test-file))))

doing

lein test

Gives me an XML test report (with no tests).

Now, trying to run the actual Riemann tests

lein rtest riemann.config.clj

prints the testing output, but does not generate XML output. Does anyone have an idea what is wrong? Does calling the Riemann main class not trigger the Junit plugin?

Thanks a lot for help!

[mcorbin]

Hi,

I don't know how lein-test-report-junit-xml works, but in the next Riemann release, you will be able to do : riemann -Dtest.output.format=junit test riemann.config. You could also use -Dtest.output.file to specify an output file. It will output the result like:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
    <testsuite name="foo-test">
        <testcase name="footest" classname="foo-test">
            <failure>expected: 1
  actual: (2)
      at: MultiFn.java:229</failure>
        </testcase>
    </testsuite>
    <testsuite name="config-test" package="riemann">
        <testcase name="foo" classname="riemann.config-test">
        </testcase>
    </testsuite>
</testsuites>

(note the MultiFn.java in the failure block, i am trying to find why it's not the good line number).

[peterneubauer]

That would be fabulous, when is the release planned?

[mcorbin]

#893 and #892 needs to be merged ;)

[mcorbin]

The release is done ;)

[peterneubauer]

Awesome, thank you so much @mcorbin !