Multiple scenarios per feature file do not show in the VS Code Test Explorer
deyanp opened this issue · 10 comments
I create a feature file with 2 scenarios, but when I looked at the VS Code/.NET Test Explorer I see only 1 test for the feature, not 1 for each scenario, as stated here:
"If the feature has multiple scenarios, add them to the same feature file. They will show up as additional tests in the test explorer. And they will need additional methods in the same feature class for execution."
Did something change recently, or I have done something wrong?
Can you please provide the feature file content and the feature class code too?
That functionality has not changed recently and the documentation is up to date, to the best of my knowledge.
Pls check https://github.com/deyanp/XunitGherkinTests in VS Code with .Net Core Test Explorer.
What I see is:
even though I have 2 scenarios in the feature file (I just copied the first scenario and changed its title, the steps are the same):
Feature: AddTwoNumbers
In order to learn Math
As a regular human
I want to add two numbers using Calculator
Scenario: Add two numbers
Given I chose 12 as first number
And I chose 15 as second number
When I press add
Then the result should be 27 on the screen
Scenario: Add three numbers
Given I chose 12 as first number
And I chose 15 as second number
When I press add
Then the result should be 27 on the screen
Just wanted to add that on the cmd line the tests are recognized ...
PS C:\Users\Deyan\TryProjects\XUnitTests> dotnet test -t
Test run for C:\Users\Deyan\TryProjects\XUnitTests\bin\Debug\netcoreapp2.2\XUnitTests.dll(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
The following Tests are available:
NormaldTests.My test
AddTwoNumbers :: Add two numbers
AddTwoNumbers :: Add three numbers
PS C:\Users\Deyan\TryProjects\XUnitTests>
And one more note ;), the .Net Core Test Explorer for VS Code is issuing the following command to find the tests in a project:
Executing dotnet vstest "c:\Users\Deyan\TryProjects\XUnitTests\bin\Debug\netcoreapp2.2\XUnitTests.dll" /ListFullyQualifiedTests /ListTestsTargetPath:"C:\Users\Deyan\AppData\Local\Temp\test-explorer-discover-jksTDu\output.txt" in c:/Users/Deyan/TryProjects/XUnitTests/
I see. Maybe that command is the issue. You need to run tests using dotnet test command. I've never tried to run it using dotnet vstest. Can you please use a .NET core CLI and run that command manually (dotnet vstest) and see the results? What does it give? Also run dotnet test and compare the results. Tell me what you find.
Is there a way to configure the vs code to issue the correct command?
I retested with .Net Core Test Explorer in VS Code, and it issues 2 commands, not 1:
Executing dotnet test -t -v=q in c:/Users/Deyan/TryProjects/XUnitTests/
Executing dotnet vstest "c:\Users\Deyan\TryProjects\XUnitTests\bin\Debug\netcoreapp2.2\XUnitTests.dll" /ListFullyQualifiedTests /ListTestsTargetPath:"C:\Users\Deyan\AppData\Local\Temp\test-explorer-discover-K4ZFFI\output.txt" in c:/Users/Deyan/TryProjects/XUnitTests/
It seems that after discovery .Net Core Test Explorer shows "? Scenario" only
but after the run it refreshes the view and displays the correct scenario names
I see. And what about when you run it from the command line? Does it work fine?
Maybe the VS code test runner behaves differently and then this issue needs to be about VS code test runner specifically. I haven't looked into that as I use regular visual studio all this time.
dotnet test works fine, it lists the individual scenarios, and tells you which one of them fails, and which not.
Do you know if it is at all possible to use the --filter arg with dotnet test, and run a single scenario (out of a feature) only?
Yes, it is possible to only execute certain scenarios. You need to use tags for that. Please take a look at this instruction: https://github.com/ttutisani/Xunit.Gherkin.Quick/blob/master/docs/tags.md
In general, you can scroll down the main page of this repo and look through "See Also" section, which might give you many tips and tricks about various capabilities of the framework.
Any questions on this? Otherwise, this ticket can be closed.