Testing not possible

Question

Testing not possible

Closed this issue 10 years ago · 28 comments

First off, I love the way you are handling tests in Mimosa.
However, I can't seem to get it to work.

In my project I created a spec at "assets\javascripts\spec\test_spec.coffee".
What works is mimosa compiling it to public\javascripts\spec\test_spec.js

When I first created a test I just called it test.coffee, which didn't trigger mimosa test to run. After that I renamed it to test_spec.coffee, and mimosa run it once, However, it errored out:

20:19:27 - Success - Compiled/copied public\javascripts\spec\test.js
20:19:28 - Success - 0 of 0 tests passed.
20:20:13 - Success - Compiled/copied public\javascripts\spec\test.js
20:20:15 - Success - 0 of 0 tests passed.
20:22:53 - Success - Compiled/copied public\javascripts\spec\test.js
20:22:55 - Success - 0 of 0 tests passed.
20:25:33 - Success - Deleted file public\javascripts\spec\test.js
20:25:33 - Success - Compiled/copied public\javascripts\spec\test_spec.js
20:25:35 - ERROR - 1 of 1 tests failed.
not ok 1 Error
    ---
        message: >
            Testem Server Error: listen EADDRINUSE

20:25:35 - Success - 0 of 0 tests passed.
20:26:48 - Success - Compiled/copied public\javascripts\spec\test_spec.js
20:26:49 - Success - 0 of 0 tests passed.

Since then, I am not able to let it run any tests again.
Did I miss anything?

here is my testcode:

define [], ->

  describe 'Satisfy', ->
    it 'should make sense', ->
      expect(0).to.equal(0)
    it 'should be working', ->
      expect('').to.not.equal(1)

Answer 1 · 2014-02-13T19:48:01.000Z

If you shut down mimosa and run it again, same error?

Answer 2 · 2014-02-13T19:52:43.000Z

If I run it again, there is no more test run:

D:\dev\satisfy> mimosa watch -s
20:44:11 - Info - No Bower installs needed.
20:44:11 - Info - Watching assets
20:44:11 - Success - Mimosa is starting your server: server.coffee
Express server listening on port 3000 in development mode
   info  - socket.io started
20:44:12 - Success - 0 of 0 tests passed.

I've updated the code here:
https://github.com/Anachron/satisfy

Answer 3 · 2014-02-13T19:55:23.000Z

Works for me!

14:54:14 - Info - No Bower installs needed.
14:54:14 - Info - Watching assets
14:54:14 - Success - Mimosa is starting your server: server.coffee
Express server listening on port 3000 in development mode
   info  - socket.io started
14:54:16 - Success - 2 of 2 tests passed.

Wipe out your .mimosa and public directory and give it another shot?

Do you have phantomjs installed properly? What OS are you on?

Answer 4 · 2014-02-13T20:00:22.000Z

Doesn't work after cleaning .mimosa

I'm on Windows 7 Ultimate x64 and downloaded phantomjs and put it here:
D:\apps\phantomjs. This path I added to my PATH-Environment.

Answer 5 · 2014-02-13T20:03:18.000Z

Given its Windows I'm not sure exactly what the problem might be. Things are working ok and the easiest assumption is something wrong with phantom.

Answer 6 · 2014-02-13T20:09:32.000Z

What I've been missing is the command "Starting Browser".

Restored question:
How does this actually load testem here?
https://github.com/dbashford/mimosa-testem-require/blob/master/assets/run-tests.js#L8

Answer 7 · 2014-02-13T20:11:42.000Z

Curious how testem is fed in there?

Testem manages that.

Curious if you can get it to run without using phantomjs.

mimosa testscript, then run that script.

Answer 8 · 2014-02-13T20:12:40.000Z

You'll need to have testem installed globally. npm install -g testem

Answer 9 · 2014-02-13T20:15:04.000Z

Yupp, the tests are passing and my console and my browser shows the correct result.

Answer 10 · 2014-02-13T20:26:21.000Z

So its definitely a phantomjs issue of some sort.

Answer 11 · 2014-02-13T20:38:06.000Z

Things are getting more and more weird:

PS D:\dev\satisfy> npm ls --depth=0 -g
D:\apps\scoop\apps\nodejs\0.10.25
├── bower@1.2.8
├── i18n@0.4.1
├── mimosa@2.1.0
├── mongoose@3.8.3
├── npm@1.3.25
├── phantomjs@1.9.7-1
└── testem@0.6.7

PS D:\dev\satisfy> mimosa watch -s
21:37:17 - Info - Starting Bower install...
21:37:17 - Info - mimosa-bower created file assets\javascripts\vendor\requirejs\
require.js
21:37:17 - Info - mimosa-bower created file assets\javascripts\vendor\jquery\jqu
ery.js
21:37:17 - Info - Cleaned temp bower output directory .mimosa\bower\bower_compon
ents
21:37:17 - Info - Watching assets
21:37:17 - Success - Compiled/copied public\javascripts\app.js
21:37:17 - Success - Compiled/copied public\javascripts\todos.js
21:37:17 - Success - Compiled/copied public\javascripts\common.js
21:37:17 - Success - Compiled/copied public\javascripts\spec\test_spec.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\jquery\jquery.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\requirejs\require
.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\backbone\1.1.0\ba
ckbone.min.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\backbone.epoxy\1.
0.5\backbone.epoxy.min.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\backbone.epoxy\1.
0.5\backbone.epoxy.js
21:37:17 - Success - Compiled/copied public\javascripts\vendor\handlebars\1.3.0\
handlebars.amd.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\jquery\2.0.3\jque
ry.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\handlebars\1.3.0\
handlebars.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\backbone.localsto
rage\1.1.7\backbone.localStorage.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\marionette\1.4.1\
backbone.marionette.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\backbone\1.1.0\ba
ckbone.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\marionette\1.4.1\
backbone.marionette.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\require\2.1.8\req
uire.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\jquery\2.0.3\jque
ry.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\raphael\2.1.2\rap
hael.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\morris\0.4.3\morr
is.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\backbone.localsto
rage\1.1.7\backbone.localStorage.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\morris\0.4.3\morr
is.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\raphael\2.1.2\rap
hael.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\marionette\1.5.1\
backbone.marionette.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\semantic\0.12.4\s
emantic.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\marionette\1.5.1\
backbone.marionette.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\handlebars\1.3.0\
handlebars.amd.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\underscore\1.5.2\
underscore.min.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\handlebars\1.3.0\
handlebars.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\underscore\1.5.2\
underscore.js
21:37:18 - Success - Compiled/copied public\javascripts\vendor\require\2.1.8\req
uire.min.js
21:37:18 - Success - Mimosa is starting your server: server.coffee
Express server listening on port 3000 in development mode
   info  - socket.io started
21:37:18 - ERROR - 0 of 0 tests failed.

Answer 12 · 2014-02-13T20:39:40.000Z

How is that more weird?

O, you have it installed via npm? Uninstall that. You should not have it installed with both NPM and via Windows PATH.

Answer 13 · 2014-02-13T20:43:43.000Z

I've just removed the windows-install and removed it from PATH, however, the result is the same:
(After cleaning .mimosa too)

21:42:13 - Success - Mimosa is starting your server: server.coffee
Express server listening on port 3000 in development mode
   info  - socket.io started
21:42:14 - ERROR - 0 of 0 tests failed.

What is weird now is that it ERRORs out, instead of saying success. Thus it should mean that it finds phantomjs, in my opinion.

Edit: FYI

PS D:\dev\satisfy> phantomjs --version
1.9.7

Answer 14 · 2014-02-13T20:50:31.000Z

You should have actually done the opposite. It doesn't work via NPM on Windows I'm afraid. Sorry!

Answer 15 · 2014-02-13T20:57:59.000Z

May I ask why it's not working? Is it because of Mimosa?
I've just done the opposite, uninstall the npm and putting it back in place:

You were right:

PS D:\dev\satisfy> mimosa watch -s
21:53:19 - Info - No Bower installs needed.
21:53:19 - Info - Watching assets
21:53:20 - Success - Mimosa is starting your server: server.coffee
Express server listening on port 3000 in development mode
   info  - socket.io started
21:54:38 - Success - 2 of 2 tests passed.

However, running this command took over one minute. That's not supposed to be normal, right?

Answer 16 · 2014-02-13T21:04:35.000Z

To be even more specific:

The process (1)

D:\apps\scoop\apps\nodejs\0.10.25\\node.exe D:\apps\scoop\apps\nodejs\0.10.25\node_modules\mimosa\bin/mimosa watch -s

will spawn (2)

cmd.exe /s /c "D:\dev\satisfy\node_modules\mimosa-testem-require\node_modules\mimosa-testem-simple\node_modules\.bin\testem ci --file D:\dev\satisfy\.mimosa\testemRequire\testem.json"

which will spawn (3)

node   "D:\dev\satisfy\node_modules\mimosa-testem-require\node_modules\mimosa-testem-simple\node_modules\.bin\\..\testem\testem.js" ci --file D:\dev\satisfy\.mimosa\testemRequire\testem.json

which will spawn (4)

phantomjs D:\dev\satisfy\node_modules\mimosa-testem-require\node_modules\mimosa-testem-simple\node_modules\testem/assets/phantom.js http://localhost:7357/9985

The last process shuts down after 1 second (finished).
Now process (3) will wait for about one minute until it sends the feedback to process (2).

Answer 17 · 2014-02-13T21:08:01.000Z

It not working via NPM on windows is a phantomjs thing. Not sure why, just know that is how it is.
The rest of those processes are likely caught up by testem. I'd look into testem on windows to see if there are any known issues. I think that 3rd process is hanging things up, the other processes are likely behaving as expected.

Answer 18 · 2014-02-13T21:24:15.000Z

I've actually founds this post about the problem
nodejs/node-v0.x-archive#3871 (comment)

Actual issue is here
nodejs/node-v0.x-archive#3584

A potential fix
https://github.com/airportyh/mocha/compare/windowsfix

Checking testem, it seems to not respect that
https://github.com/airportyh/testem/blob/master/lib/launcher.js

Well, this really sucks. I can't use phantomjs-testing now.

Edit: I hope this is actually the problem I'm having...

Answer 19 · 2014-02-13T21:33:04.000Z

You can turn off the testing during watch and just use the test script. I actually suggest you use testem directly (via the script) in another terminal window while you have mimosa running. Using testem directly is the way to actually WRITE your tests. Having it running things while you code is the way to go when you are writing code to make sure you don't break things.

Answer 20 · 2014-02-13T21:45:07.000Z

Yupp, I've added

  testemRequire:
    executeDuringWatch: false

to mimosa-config.coffee and tests are gone for watching now.
Will use the bat-file so far, but I would love to be able to connect it to mimosa too.

(Will debug the problem a little longer and may create a issue on testem)

Answer 21 · 2014-02-13T21:49:28.000Z

Going to close this issue out. If your research makes you think this module needs to change, feel free to open a new issue.

Answer 22 · 2014-02-13T22:05:04.000Z

I've found out that the mimosa-module is not using the latest testem-version:
https://github.com/dbashford/mimosa-testem-simple/blob/master/package.json#L26

And since that module is local, the global module will be ignored.
Maybe that's connected to the problem?

Answer 23 · 2014-02-13T22:07:28.000Z

Give it a shot?

Just update the version in that package.json and run npm install in that directory.

Answer 24 · 2014-02-15T13:47:38.000Z

Seems to be an issue connected with mimosa:
testem/testem#345 (comment)

Could you take a look?

Answer 25 · 2014-02-15T15:33:53.000Z

@Anachron I'm sorry but can you explain why you think the delay observed on Windows relates to joyent/node#3584?

That issue has to do with process.exit() being called before the child process stdout and stderr were flushed on Windows, regardless of whether it was spawned with spawn or exec. It is a core Node issue that unfortunately has no known fix yet.

It used to be a problem in that testem ci's output from phamtomjs would be truncated as a result. A workaround that involved draining stdout and stderr was suggested for the Node issue. It has been applied to Testem 8 months ago and is still present in the latest version of Testem:
https://github.com/airportyh/testem/blob/master/lib/clean_exit.js

So as far as the truncation issue goes, that is no longer a problem.

The delay issue with the subprocess exiting while the parent process still waits for it is something I have seen reported in the past. I just can't for the life of me remember from where. But it doesn't seem related to the Node issue you referred to.

Am I missing something?

Answer 26 · 2014-02-15T15:38:01.000Z

@brzpegasus yes you are right, I thought the delay would come from the parent-process waiting for childs-output to clear which will not clear because of a node-problem.

But the more I think about it, the less it seems to fit to this situation.

Answer 27 · 2014-02-15T16:33:16.000Z

@Anachron Also, you were wondering why the module didn't work when phantomjs is installed via npm on Windows. This is why:

joyent/node#2318

In short, child_process.spawn() only works with .exe files. When you install phantomjs from the main site, you're putting that executable in your PATH, so there's no issue there. However, when you install phantomjs from npm, phantomjs now refers to the phantomjs.cmd that gets installed in your ${user.home}/AppData/Roaming/npm folder, and spawn() doesn't work with batch files. So what you get is a spawn ENOENT error.

That's yet another unresolved Node + Windows issue =).

Answer 28 · 2014-02-15T16:57:19.000Z

Thanks for your amazing post, @brzpegasus! There actually seems to be a workaround for that, but joyent doesn't seem to update node to implement it :/