Rake Tasks not running in predicable order

[jkthorne]

I am trying to run some rake tasks in order and they seem to be executing out of order in some weird way.
This issue is related to rubocop/rubocop#1455 (comment)

task :test do
  Rake::Task['test:all'].invoke
  Rake::Task['rubocop'].invoke
end

RuboCop::RakeTask.new do |task|
  task.fail_on_error = false
end

$ bundle exec rake --dry-run
** Invoke default (first_time)
** Invoke test (first_time)
** Invoke test:all (first_time)
** Execute (dry run) test:all
** Invoke rubocop (first_time)
** Execute (dry run) rubocop
** Execute (dry run) test
** Execute (dry run) default

somehow rubocop fails before the test run.

$ bundle exec rake
<Rake::Task default => [test]>
2014-11-21T00:41:35Z 89007 TID-ouqfgo0z0 INFO: Sidekiq client with redis options {:url=>"redis://redis-1:6379/2", :namespace=>"squaw"}
Running RuboCop...
Inspecting 108 files
.................................................................................C.....................C....

Offenses:

Rakefile:10:1: C: Extra blank line detected.

[drbrain]

Rake can only make guarantees about order when you use dependencies.

task test: %w[test:all rubocop]

If rubocop must be run after test:all add such a dependency.

[drbrain]

Thanks for pinging me via slack.

It turns out that your problem is due to the way rails creates its test tasks:

  desc "Run tests quickly by merging all types and not resetting db"
  Rails::TestTask.new(:all) do |t|
    t.pattern = "test/**/*_test.rb"
  end

https://github.com/rails/rails/blob/v4.1.8/railties/lib/rails/test_unit/testing.rake#L19-L22

Here Rails uses Rails::TestTask for the test:all target which will load all test files.

    def define
      task @name do
        if ENV['TESTOPTS']
          ARGV.replace Shellwords.split ENV['TESTOPTS']
        end
        libs = @libs - $LOAD_PATH
        $LOAD_PATH.unshift(*libs)
        file_list.each { |fl|
          FileList[fl].to_a.each { |f| require File.expand_path f }
        }
      end
    end
  end

https://github.com/rails/rails/blob/v4.1.8/railties/lib/rails/test_unit/sub_test_task.rb#L106-L118

But unlike Rake::TestTask, which immediately runs the tests, Rails::TestTask only requires the files necessary to run the tests then relies on the at_exit handler in Minitest to run the tests. This means rake dependencies are completely ignored for running tests.

I feel this is a bug in rails. Can you create one and reference this issue?