Tests fail in tinytest, but not spacejam
Closed this issue · 6 comments
Man, I'm really looking for a predictable command-line testing tool for packages, but getting disheartened.
The first package I wrote and tried to validate this tool on, I get only the following output on the console (did it pass, fail, error?), and a 0 exit code regardless of whether tinytest sees the failure.
spacejam test-packages packages/schedule-the-things\:data/
spacejam: spawning meteor
[[[[[ Tests ]]]]]
=> Started proxy.
=> Started MongoDB.
spacejam: meteor mongodb is ready
spacejam: killing meteor
~/src/deanius/schedule-the-things (master *) $ echo $?
0
Package source:
https://s3.amazonaws.com/www.chicagogrooves.com/spacejam-debug.zip
By contrast, arunoda's phantom jS runner in this case gives me something I can see is working, and which actually works:
node start_test.js
I20150426-22:17:02.662(-5)? test-in-console listening
starting testing...
Running Meteor tests in PhantomJS... http://localhost:10015/
S: tinytest - data : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest"],"test":"data","events":[{"sequence":0,"type":"fail","details":{"type":"assert_equal","expected":"true","actual":"false","not":false}}],"server":true}
C: tinytest - data : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest"],"test":"data","events":[{"sequence":0,"type":"fail","details":{"type":"assert_equal","expected":"true","actual":"false","not":false},"cookie":{"name":"data","offset":0,"groupPath":["tinytest"],"shortName":"data"}}]}
~~~~~~~ THERE ARE FAILURES ~~~~~~~
passed/expected/failed/total 0 / 0 / 2 / 2
##_meteor_magic##state: done
Is there any insight you have into why this package's failures aren't detected by Spacejam?
spacejam uses tinytest, it is only a launcher for meteor test-packages and for phantomjs. arunoda's tool is the same, but it is a script instead of an npm package. spacejam outputs the same, since this ouptut comes from phantomjs in both tools. I downloaded your package and ran the tests using spacejam and got the complete expected output. What OS r u on?
OSX Yosemite. Thanks for checking that out.. Do you suggest any next steps for me to debug ?
Can you let me know the version of spacejam you're using, the command and if you're using ROOT_URL or MONGO_URL? If you do, you will need to set them using spacejam command line flags, since spacejam ignores them to not run tests on same mongodb as your app's one. Same for ROOT_URL
Also, I'll run it on os x tomorrow.