Can't run single test file with `spring: true` set

[chrisnicola]

For some reason extra tests are being run whenever spring: true is set.

With guard:

12:42:49 - INFO - Running: test/models/user_test.rb
Run options: --seed 25521

# Running:

.....................................................................................................

Finished in 8.738868s, 11.5576 runs/s, 23.6873 assertions/s.

101 runs, 207 assertions, 0 failures, 0 errors, 0 skips

Directly:

bin/rake test TEST=test/models/user_test.rb
Run options: --seed 21314

# Running:

................................

Finished in 2.014514s, 15.8847 runs/s, 27.7983 assertions/s.

32 runs, 56 assertions, 0 failures, 0 errors, 0 skips

[chrisnicola]

Sorry some digging has traced this back to some local changes to the rake test task.

[richardvenneman]

@chrisnicola I haven't made any local changes to the test rake task but I do have the same issue. Could you please share what you've done to make it work as expected?

[chrisnicola]

I can't recall, but I believe we had customized the task to run some extra test folders and there was a mistake.

[gshaw]

@richardvenneman did you ever resolve your problem? I'm having the same issue as well.

[e2]

@gshaw - run guard with the -d option: bundle exec guard -d. Guard should intercept the Kernel.system command and show what exactly it's running.

Then, the next step would be to run that exact command outside Guard.

[gshaw]

@e2, I did. When you run with spring: true guard is running with bin/rails rake and passing all the test paths separated only by whitespace. E.g,

When spring: true

bin/rake test test/models/setting_test.rb test/models/user_test.rb

When spring: false it uses a completely different command line:

bundle exec ruby -I"test" -I"spec" -r bundler/setup -r minitest/autorun -r ./test/models/setting_test.rb -r ./test/models/user_test.rb -e "" --

If only one file is passed when spring: true it actually works but if multiple files are passed all the tests are executed. You can duplicate both scenarios on the command line outside of Guard.

The problem is that I'm not sure how to pass multiple tests to bin/rake test to take advantage of running them under spring.

[e2]

@gshaw - I'm not a minitest user (too frustrating in my experience), but I might have a few hints:

You can duplicate both scenarios on the command line outside of Guard.

Then it isn't a Guard::Minitest issue. Not until the problem is understood in a way where Guard::Minitest can help.

bin/rake test test/models/setting_test.rb test/models/user_test.rb

I believe it should go something like this: bin/rake test TEST='test/models/setting_test.rb test/models/user_test.rb' as suggested here:

https://github.com/rails/rails/blob/4-2-stable/railties/lib/rails/test_unit/sub_test_task.rb#L20

(Though I'm not sure how minitest handles ENV['TEST'] with multiple files).

So it's running all the tests because you're not setting the TEST environment variable.

It's just that minitest expects environment variables, not arguments, e.g. TESTOPTS= should be recognized as well.

I'm also not sure about how forking plays here. With forking environment variables are usually that of the original process (e.g. spring in this case), unless they're set manually somewhere.

Add some puts statements in the railties sources files to see what's being passed and how it's handled.

[e2]

If it's a bug in how bin/rake is called, I can make a quick fix-and-release if I know what to change.

[gshaw]

OK, it took a bit of looking but the problem I believe is how Rails 5 is favoring bin/rails test instead of bin/rake test (note the use of rails instead of rake). I can solve the problem with the current version of guard-mintiest by using this in my Guardfile.

guard "minitest", spring: "bin/rails test" do
  ...
end

It might be a good idea to check for Rails 5 and use by default bin/rails test instead of bin/rake test but I'm not sure of a good way to do that inside the guard-mintiest gem.

[e2]

@gshaw - that makes sense, since Rake uses/handles arguments differently.

Rails should already be activated as a gem, so detection is possible - Guard pretty much tries to use bundler if possible. Though minitest autoloads plugins through rubygems - so that would take testing.

So testing for Rails in activated gems could be enough. Another idea is the new Rails 5 API - lots has changed on the testing side, so it's possible to just reference classes/methods new in Rails 5 (e.g. railties related to testing).

Does the TEST= variable work for multiple files? If so, then maybe just the command-line can be changed, so that spring+rake to compose the TEST= parameter?

[gshaw]

The TEST= variable does work for multiple files. I have everything working well with Rails 5 and latest guard and spring after I explicitly passed in the command to use spring: "bin/rails test".

At least for now adding a conditional to https://github.com/guard/guard-minitest/blob/master/lib/guard/minitest/runner.rb#L166 to check if Rails 5 is activated and if so use bin/rails test instead of bin/rake test would be enough.

I'd save the deeper integration with the Rails 5 API at least until the final version is out and then only if there is a real need for the extra coupling.

[e2]

I think a better patch is to not be Rails-5 "aware", but to actually compose the TEST= parameter, e.g. replacing: https://github.com/guard/guard-minitest/blob/2db91fb/lib/guard/minitest/runner.rb#L173

with:

cmd_parts + "TEST='#{paths.join(' ')}'"

though I'm not sure if that will be passed down properly.

[burlesona]

Just want to second @e2's suggestion. I'm stuck on an older version of Rails and having this exact problem, with spring turned on it's not possible to run only one test.