Path to failing test instead of title
firedev opened this issue · 4 comments
We're running a suite of tests with Jenkins and it's a bit inconvenient to gather failing tests in the log files. Is it possible to get something like ./spec/spec_file.rb:123 instead of the description of the failing test in the output?
Thanks.
The full path to the example with a line number should be available in the file attribute of each unit test entry. What are you displaying your junit results within?
Jenkins, here is what I have in XML:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="17" failures="1" errors="0" time="3.754125" timestamp="2016-09-03T03:43:24+00:00">
<properties/>
<testcase classname="spec.lib.Test.rack.apps.sales.orders_spec" name="Test::Rack::Apps::Sales::Orders authorization GET /:id with sales.order.view permission succeeds" time="0.147581">
<failure message="expected `#<Rack::MockResponse:0x00000007ac13c0 @original_headers={"Content-Type"=>"application/json", "X-Content-Type-Options"=>"nosniff"}, @errors="", @body_string=nil, @status=500, @header={"Content-Type"=>"application/json", "X-Content-Type-Options"=>"nosniff", "Content-Length"=>"172"}, @chunked=false, @writer=#<Proc:0x00000007ac0b00@/var/lib/gems/2.1.0/gems/rack-1.4.5/lib/rack/response.rb:28 (lambda)>, @block=nil, @length=172, @body=["{\"error\":\"internal_error\",\"message\":\"Test encountered an internal error. Please try your request again. If the problem persists, contact your system administrator.\"}"]>.ok?` to return true, got false" type="RSpec::Expectations::ExpectationNotMetError">
<![CDATA[expected `#<Rack::MockResponse:0x00000007ac13c0 @original_headers={"Content-Type"=>"application/json", "X-Content-Type-Options"=>"nosniff"}, @errors="", @body_string=nil, @status=500, @header={"Content-Type"=>"application/json", "X-Content-Type-Options"=>"nosniff", "Content-Length"=>"172"}, @chunked=false, @writer=#<Proc:0x00000007ac0b00@/var/lib/gems/2.1.0/gems/rack-1.4.5/lib/rack/response.rb:28 (lambda)>, @block=nil, @length=172, @body=["{\"error\":\"internal_error\",\"message\":\"Test encountered an internal error. Please try your request again. If the problem persists, contact your system administrator.\"}"]>.ok?` to return true, got false
./spec/lib/Test/rack/apps/sales/orders_spec.rb:66:in `block (5 levels) in <top (required)>'
./spec/spec_helper.rb:94:in `block (3 levels) in <top (required)>'
/vagrant/core/lib/sequel/extensions/before_commit.rb:22:in `block in _transaction'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/database/transactions.rb:111:in `_transaction'
/vagrant/core/lib/sequel/extensions/before_commit.rb:21:in `_transaction'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/database/transactions.rb:93:in `block in transaction'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/database/connecting.rb:229:in `block in synchronize'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/connection_pool/threaded.rb:104:in `hold'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/database/connecting.rb:229:in `synchronize'
/var/lib/gems/2.1.0/gems/sequel-4.0.0/lib/sequel/database/transactions.rb:86:in `transaction'
./spec/spec_helper.rb:94:in `block (2 levels) in <top (required)>'
/var/lib/gems/2.1.0/gems/rspec-retry-0.4.5/lib/rspec/retry.rb:98:in `block in run'
/var/lib/gems/2.1.0/gems/rspec-retry-0.4.5/lib/rspec/retry.rb:88:in `loop'
/var/lib/gems/2.1.0/gems/rspec-retry-0.4.5/lib/rspec/retry.rb:88:in `run'
/var/lib/gems/2.1.0/gems/rspec-retry-0.4.5/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
/var/lib/gems/2.1.0/gems/rspec-retry-0.4.5/lib/rspec/retry.rb:22:in `block (2 levels) in setup']]>
</failure>
</testcase>
</testsuite>
Oh right, sorry. It's been a while since I've used Jenkins or JUnit!
Afaik the Jenkins JUnit support displays nested trees of packages and test cases, and file paths as class names break the presentation. The problem is that JUnit is made for Java, which uses package namespaces and class names for organization, not file paths like Ruby/Rspec. Is there a better way to shove this information into JUnit for jenkins? Is the stack trace in the failure output not enough?
The classname based on file path combined with stracktrace should be enough for this. There's also stronger support for shared example locations coming soon.