Scenario are not being registered

Question

Scenario are not being registered

Closed this issue 5 years ago · 13 comments

This is really weird, I don't know how this is triggered, maybe for the amount of scenarios I have. Take a peek at my register.feature file, The last scenario has a tag named @only, but, when I tell TestCafe-Gherkin to run that specific tag, it tells me there is no tests for it to run.

If I move that scenario higher up in the list, then it is able to locate it and run it without any issues.

This is happening to 2 of my feature files now. I'm I doing something wrong?

Feature: User is able to register

  @register @failingRegister
  Scenario: Should NOT be able to register without filling in any information
    Given I am at the register page
    When I am register with email "" and password "" with confirm password ""
    Then I should NOT be able to press the signup button

  @register @failingRegister
  Scenario: Should NOT be able to register without filling in password
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "" with confirm password ""
    Then I should NOT be able to press the signup button

  @register @failingRegister
  Scenario: Should NOT be able to register without filling in confirm password
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "AmazingPass!2387" with confirm password ""
    Then I should NOT be able to press the signup button

  @register @failingRegister
  Scenario: Should NOT be able to register if password is fewer then 8 characters
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "aD$3!2j" with confirm password "aD$3!2j"
    Then I should NOT be able to press the signup button

  @register @successfullRegistration @only
  Scenario: Should be able to register an account and continue without verifying email
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "AmazingPass!2387" with confirm password "AmazingPass!2387"
    Then I should be able to press the signup button
    Then I should be able to see successfull signup screen
    Then I should be able to continue to account setup

  @register @failingRegister
  Scenario: Should NOT be able to register with miss matching passwords
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "ThisWas4321OneThing" with confirm password "ThisWasAnotherThing"
    Then I should see must be the same as password
    Then I should NOT be able to press the signup button

  @register @successfullRegistration @only
  Scenario: Should be able to register an account and continue without verifying email
    Given I am at the register page
    When I am register with email "bob@bobsTown.com" and password "AmazingPass!2387" with confirm password "AmazingPass!2387"
    Then I should be able to press the signup button
    Then I should be able to see successfull signup screen
    Then I should be able to verify my email
    Then I should be able to continue setting up my account

If I run all test without using tags, then it is ignoring the test as well.

TestCafe version: 1.4.1
gherkin-testcafe version: 2.4.0

No tests to run. Either the test files contain no tests or the filter function is too restrictive

Answer 1 · 2019-08-23T21:38:49.000Z

Deleted my node_modules and re-installed it, seemed to have fixed it

Answer 2 · 2019-08-27T15:56:00.000Z

it is keep on happening, even after removing and then re-installing node_modules packages.

This is my latest Feature file:

Feature: Sales Guidance Happy Path

  @smokeTest @rolesAndPermission @roleSuperUser
  Scenario: User with company Stakeholder roll is able to access all aspects of Dealer Support - (auth0 role name: superuser)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will be able to see and access Configuration
    Then user will be able to see and access "edit-products"
  # Then user will be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleSales
  Scenario: User with company Sales Rep roll is able to access to help manage and sell to external dealers - (auth0 role name: sales)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will NOT be able to see and access "sales-guidance"
    Then user will be able to see and access "pricing-helper"
    Then user will NOT be able to see and access  "warranty-claim"
    Then user will NOT be able to see and access  "begin-registration"
    Then user will NOT be able to see and access  "product-manuals"
    Then user will NOT be able to see and access  "configuration"
    Then user will NOT be able to see and access  "edit-products"
  # Then user will NOT be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleDealer
  Scenario: User with Dealer User roll is able to access to help buy products from company to sell to end consumers - (auth0 role name: dealer)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will will be to see and access Configuration
    Then user will NOT be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleDealerAdmin
  Scenario: User with Dealer Admin roll is able to access to help manage Dealer-specific administrative tasks - (auth0 role name: dealer-admin)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will be able to see and access Configuration
    Then user will be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

If I remove the last 2 scenarios, I am able to run the two first ones. If I add the 2 last scenarios back... it isn't picking it up.

Answer 3 · 2019-08-27T21:39:19.000Z

Sorry to hear that the error still persists.

Could you verify for me if the error still occurs if you only load that specific feature file (no other files)?

Answer 4 · 2019-08-27T22:20:33.000Z

seems to work when I removed all my other feature files except that one

Answer 5 · 2019-08-27T22:23:25.000Z

actually, it seems to still happen, just added a new tag and I can't run it.

Removed all other .feature files in my project, was able to run @roleSuperUser
Then I added a new tag @dennis and I wasn't able to run that one.

This is the current feature file:

Feature: Making sure that all roles have only access to what they should have access too

  @smokeTest @rolesAndPermission @roleSuperUser
  Scenario: User with company Stakeholder roll is able to access all aspects of Dealer Support - (auth0 role name: superuser)
    Given I am at the welcome screen
    When dashboard has loaded
    Then I will be able to see and access "product-education"
    Then I will be able to see and access "sales-guidance"
    Then I will be able to see and access "pricing-helper"
    # Then I will be able to see and access "warranty-claim"
    Then I will be able to see and access Begin Registration
    Then I will be able to see and access "product-manuals"
    Then I will be able to see and access Configuration
    Then I will be able to see and access "edit-products"

  @smokeTest @rolesAndPermission @roleSales
  Scenario: User with company Sales Rep roll is able to access to help manage and sell to external dealers - (auth0 role name: sales)
    Given I am at the welcome screen
    When dashboard has loaded
    Then I will be able to see and access "product-education"
    Then I will NOT be able to see and access "sales-guidance"
    Then I will be able to see and access "pricing-helper"
    Then I will NOT be able to see and access  "warranty-claim"
    Then I will NOT be able to see and access  "begin-registration"
    Then I will NOT be able to see and access  "product-manuals"
    Then I will NOT be able to see and access  "configuration"
    Then I will NOT be able to see and access  "edit-products"

  @smokeTest @rolesAndPermission @roleDealer
  Scenario: User with Dealer User roll is able to access to help buy products from company to sell to end consumers - (auth0 role name: dealer)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will will be to see and access Configuration
    Then user will NOT be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleDealerAdmin
  Scenario: User with Dealer Admin roll is able to access to help manage Dealer-specific administrative tasks - (auth0 role name: dealer-admin)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will be able to see and access Configuration
    Then user will be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleSuperI @adminTool @dennis
  Scenario: User with company Stakeholder roll is able to access the Admin Tool - (auth0 role name: superuser)
    Given I am at the login screen for the Admin Tool
    When I am logged in
    Then I am able to see Admin Tool dashboard

  @smokeTest @rolesAndPermission @roleSales @adminTool
  Scenario: User with company Stakeholder roll is NOT able to access the Admin Tool - (auth0 role name: sales)
    Given I am at the login screen for the Admin Tool
    When I am logged in
    Then I see notification to contact admin to access the Admin Tool

Answer 6 · 2019-08-28T06:54:35.000Z

Well, I can't reproduce this error. I tried

gherkin-testcafe chrome example.feature: It ran all six tests
gherkin-testcafe chrome example.feature --tags @dennis: It ran the fifth test
gherkin-testcafe chrome example.feature --tags @dennis,~@roleSuperI: It did not find tests
gherkin-testcafe chrome example.feature --tags ~@roleSuperI: It ran all test except the fifth one

This all seems correct to me.

Answer 7 · 2019-08-28T15:53:58.000Z

Seems to work for me when I am running: gherkin-testcafe chrome example.feature

I am using testCafe runner to run my tests, easier to add and remove tags and what not.

I start my tests with: node src/tests/e2e/testcafe.js

This is my testcafe.js file:

const createTestCafe = require('gherkin-testcafe');
const fs = require('fs');
let testcafe = null;
const program = require('commander');

program
  .option(
    '-b, --browser <browser>',
    'What browser would you like to use? ex "chrome", "chrome:headlesss", multi - ["chrome", "safari"]'
  )
  .option(
    '-t, --tags <tag>',
    'Which tags to run? ex "@login" or "@login","@logOut" (NO space between tags). To exlude Tag(s) add "~" infront of the tag name ex "~@login"'
  )
  .option(
    '-s, --state <state>',
    'Run it "local" or "ci"? "local" is a quick way to just run the tests and make sure everything works. It will start up a local web server.'
  )
  .option(
    '-c, --concurrency <concurrency>',
    'Run it "local" or "ci"? "local" is a quick way to just run the tests and make sure everything works. It will start up a local web server.'
  )
  .option(
    '-v, --video <video>',
    'Record the test and see what is going on? Add "--video true" and there you go! Recording just for you!'
  );

program.parse(process.argv);

function readTestCafeConfig() {
  const configData = fs.readFileSync('.testcaferc.json', 'utf8');
  const js = JSON.parse(configData);
  return getValues(js, 'src');
}

function getValues(obj, key) {
  var objects = [];

  for (var i in obj) {
    if (!obj.hasOwnProperty(i)) continue;
    if (i === key) {
      objects.push(obj[i]);
    }
  }
  return objects;
}

function selectBrowser() {
  if (program.browser) {
    return program.browser;
  } else {
    return 'chrome';
  }
}

createTestCafe('localhost', 1337, 1338)
  .then(tc => {
    testcafe = tc;
    const runner = testcafe.createRunner();
    let concurrencyLevel = 1;

    runner.browsers(selectBrowser());

    if (program.state === 'ci') {
      runner.startApp('serve -s build -l 3000', 5000);
    }
    if (program.state === 'local') {
      runner.startApp('npm start', 20000);
    }
    if (program.tags) {
      runner.tags(program.tags);
    }
    if (program.concurrency) {
      concurrencyLevel = parseInt(program.concurrency, 10);
    }
    if (program.video) {
      runner.video('test-reports/videos');
    }

    return runner
      .src([readTestCafeConfig()])
      .reporter([
        'spec',
        {
          name: 'json',
          output: './test-reports/report.json'
        },
        {
          name: 'xunit',
          output: './test-reports/report.xml'
        }
      ])
      .screenshots('./test-reports/screenshots', true)
      .concurrency(concurrencyLevel)
      .run({
        selectorTimeout: 20000,
        assertionTimeout: 6000,
        pageLoadTimeout: 6000
      });
  })
  .then(failedCount => {
    console.log('Tests failed: ' + failedCount);
    testcafe.close();
    process.exit(failedCount ? 1 : 0);
  });

Answer 8 · 2019-08-28T17:07:03.000Z

It was working when I came into the office this morning, after changing some in my stepDefinition files, it stopped working with the tags. Running exact same command as I did previous.

Is there a way to get more debugging out?

Answer 9 · 2019-08-28T17:27:42.000Z

The test failed for me when I even run just the specific file:

node_modules/.bin/gherkin-testcafe chrome src/tests/e2e/tests/rolesAndPermission/rolesAndPermission.feature
The "src" option from the configuration file will be ignored.
ERROR No tests to run. Either the test files contain no tests or the filter function is too restrictive.

Answer 10 · 2019-08-28T20:02:40.000Z

If I split my tests into different feature files, then it seems to be running without an issue.

Example:
roles1.feature

Feature: Making sure that all roles have only access to what they should have access too

  @smokeTest @rolesAndPermission @roleSuperUser
  Scenario: User with company Stakeholder roll is able to access all aspects of Dealer Support - (auth0 role name: superuser)
    Given I am at the welcome screen
    When dashboard has loaded
    Then I will be able to see and access "product-education"
    Then I will be able to see and access "sales-guidance"
    Then I will be able to see and access "pricing-helper"
    # Then I will be able to see and access "warranty-claim"
    Then I will be able to see and access Begin Registration
    Then I will be able to see and access "product-manuals"
    Then I will be able to see and access Configuration
    Then I will be able to see and access "edit-products"

  @smokeTest @rolesAndPermission @roleSales
  Scenario: User with company Sales Rep roll is able to access to help manage and sell to external dealers - (auth0 role name: sales)
    Given I am at the welcome screen
    When dashboard has loaded
    Then I will be able to see and access "product-education"
    Then I will NOT be able to see and access "sales-guidance"
    Then I will be able to see and access "pricing-helper"
    Then I will NOT be able to see and access  "warranty-claim"
    Then I will NOT be able to see and access  "begin-registration"
    Then I will NOT be able to see and access  "product-manuals"
    Then I will NOT be able to see and access  "configuration"
    Then I will NOT be able to see and access  "edit-products"

roles2.feature

Feature: Making sure that all roles have only access to what they should have access too

  @smokeTest @rolesAndPermission @roleDealer
  Scenario: User with Dealer User roll is able to access to help buy products from company to sell to end consumers - (auth0 role name: dealer)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will will be to see and access Configuration
    Then user will NOT be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

  @smokeTest @rolesAndPermission @roleDealerAdmin
  Scenario: User with Dealer Admin roll is able to access to help manage Dealer-specific administrative tasks - (auth0 role name: dealer-admin)
    Given I am at the welcome screen
    When dashboard has loaded
    Then user will be able to see and access "product-education"
    Then user will be able to see and access "sales-guidance"
    Then user will NOT be able to see and access "pricing-helper"
    # Then user will be able to see and access "warranty-claim"
    Then user will be able to see and access Begin Registration
    Then user will be able to see and access "product-manuals"
    Then user will be able to see and access Configuration
    Then user will be able to see and access "edit-products"
    Then user will NOT be able to see and access "admin-tool"

roles3.feature

Feature: Making sure that all roles have only access to what they should have access too

  @smokeTest @rolesAndPermission @roleSuperI @adminTool @dennis
  Scenario: User with company Stakeholder roll is able to access the Admin Tool - (auth0 role name: superuser)
    Given I am at the login screen for the Admin Tool
    When I am logged in
    Then I am able to see Admin Tool dashboard

  @smokeTest @rolesAndPermission @roleSales @adminTool
  Scenario: User with company Stakeholder roll is NOT able to access the Admin Tool - (auth0 role name: sales)
    Given I am at the login screen for the Admin Tool
    When I am logged in
    Then I see notification to contact admin to access the Admin Tool

Now npm run e2e:test -- -t @rolesAndPermission works without an issue

Answer 11 · 2019-08-29T07:51:52.000Z

I just published v2.4.1 which updates gherkin to a new version.
Could you try if this fixes your issue? It seems to me that your issue comes from a wrong parsing of the feature files, so maybe they changed something that magically helps. :D

Is there a way to get more debugging out?

Unfortunately, no. You could run your tests in debug mode and set breakpoints in the gherkin-testcafe files, but I doubt that it would tell you much if you do not know exactly how gherkin-testcafe works.

I'm thinking about hooking into the --dev flag to generate more debug info. At the moment I don't have the time to do this tough.

Answer 12 · 2019-08-29T15:08:20.000Z

Thanks Lukas!
It seems to have fixed it, I will test around today and if I can't make the issue to occur again, I close this issue :)

Answer 13 · 2019-08-29T23:52:05.000Z

Looking good, closing this issue :)