chef-boneyard/minitest-chef-handler

chef run is not failed when a minitest has a syntax error

fnoeding opened this issue · 0 comments

minitest-chef-handler version: 1.1.0

When a minitest contains a syntax error, the minitest handler is simply aborted. It will not fail the chef run as expected.

Steps to reproduce:

  • create a minitest file that contains a error that's triggered upon require, e.g. undefined variable or a syntax error
  • run chef
  • see that chef reports an exception (see below)
  • the chef run will complete successfully

Expected behaviour:

  • the chef run will be failed with an error message

Chef output when this happens:

[...]
Report handler MiniTest::Chef::Handler raised #<NameError: undefined local variable or method `foo' for main:Object>
ERROR: /[...]/install_test.rb:3:in `<top (required)>'
ERROR: /opt/chef/embedded/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
ERROR: /opt/chef/embedded/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler/lookup.rb:14:in `block (2 levels) in require_test_suites'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler/lookup.rb:14:in `each'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler/lookup.rb:14:in `block in require_test_suites'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler/lookup.rb:13:in `each'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler/lookup.rb:13:in `require_test_suites'
ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/minitest-chef-handler-1.1.0/lib/minitest-chef-handler.rb:27:in `report'

It seems like errors during require_test_suites are not caught leading to this behaviour.