[Feature] BDD implementation with Playwright test runner
ShanmukhaGajula opened this issue · 170 comments
Our team has a case to use the BDD test, Playwright has any plan to add their own BDD implementation using the playwright test runner? That would be great if to have.
This can help many people who want to use BDD along with Playwright. That's where cucumber became popular 😉
@1996phani would https://codecept.io/playwright/ serve your needs?
BDD implementation with Playwright test runner is what am also looking for..
It would be great to have BDD with Playwright
@1996phani would https://codecept.io/playwright/ serve your needs?
@unlikelyzero, The package you're mentioning has been deprecated.
I am looking for something kind of Native support from the Playwright
Let's collect upvotes. Please upvote the OP!
Yeah BBD is a must thing!
It would be helpful ,if we have BDD along with Playwright.
BDD support is a perk
bdd is a good feature to have.. i am using in selenium java.. i need it in playwright to migrate
I'm looking for bdd too!
Good to have BDD with playwright
Having BDD with Playwright will be helpful
@1996phani would https://codecept.io/playwright/ serve your needs?
@unlikelyzero, The package you're mentioning has been deprecated.
I am looking for something kind of Native support from the Playwright
@ShanmukhaGajula where does it say that?
@1996phani would https://codecept.io/playwright/ serve your needs?
@unlikelyzero, The package you're mentioning has been deprecated.
I am looking for something kind of Native support from the Playwright@ShanmukhaGajula where does it say that?
I see a pop-up yesterday when I opened that link. I think they've removed now. However, I see playwright test runner and reporting is very good and hence requesting Playwright team for that feature within playwright itself.
BDD is always good to have
Yes, Looking for BDD to implement playwright in our project.
BDD is a must-have feature for us too! It really helps.
Hope playwright will add this BDD feature..
i was also looking for same.. BDD will really add value
I'm looking for BDD too!
Looking forward to have BDD feature with Playwright,
It would be great to have BDD with Playwright
It would be helpful ,if we have BDD along with Playwright.
It would be nice to have BDD along with Playwright.
BDD with Playwright! That sounds great. Looking forward to it.
It would be good to have the BDD feature, looking forward for it
@pavelfeldman Looks like 90% of the comments and upvotes are well planned, just digging through the user profile, most of the users just opened GitHub account to upvote this request.
May be one particular group of users really need this!
Playwright has already got support to write BDD tests using cucumber.js(dependency = "@cucumber/cucumber": "^7.3.1",). I am just wondering whether PlaywrightTestConfig (dependency = '@playwright/test') is supported in BDD tests(GIVEN, WHEN, THEN because the global configuration (const config: PlaywrightTestConfig = {
use: {.......) is not being picked automatically as written in https://playwright.dev/docs/test-configuration#global-configuration eventhough I am using playwright test runner.
Looks like the PlaywrightTestConfig works only with 'test' block in .ts or .js file. Can somebody confirm this how to call PlaywrightTestConfig in BDD tests using playwright test runner.
Cucumber integration with playwright. Looking forward for it.
Upvote! Implementation of BDD will be great!!!
Definitely a needed feature.
Cucumber integration with playwright. +1
An official cucumber Playwright adapter is needed to use all the features of playwright which works on 'test' block i.e. with @playwright/test test runnier.
Please see the comment of @mxschmitt:
Definitely needed. Integrating with e.g. Cucumber as the test runner is hacky.
Do you know when the cucumber integration will be done? This is really needed as not being able to run the tests with playwright test runner is really limiting
Would be really great if we have an official support for BDD, for syntaxes like Cucumber, etc.
We are also very much waiting for bdd support in playwright.
Now we use cucumber runner and playwright. But I really wanted to be able to also use plywright.config in this case
Upvote x1000
+1 for adding Playwright Test Cucumber integration
Pls consider the Playwright/Test-Cucumber integration
a playwright native bdd test runner would be great
It would be great to have native bdd in playwright test runner
It would be awesome to use Playwright Test with Cucumber BDD. I am upvoting behalf of all teams in our company as Test Architect.
Adding votes on behalf of my company for this feature.
Are there any plans to implement this feature request?
@kumars3 is perfectly right with #11975 (comment). It's not completely necessary to implement a BDD framework for Playwright however hooks into Playwrights' initialisation need to be exposed so that the Playwright context can be initialised. I've done some digging to try and figure out a way and came up with nothing.
Does anyone know of a workaround here so in setups within other test runners' @playwright/test context can be initialised?
@kumars3 is perfectly right with #11975 (comment). It's not completely necessary to implement a BDD framework for Playwright however hooks into Playwrights' initialisation need to be exposed so that the Playwright context can be initialised. I've done some digging to try and figure out a way and came up with nothing.
Does anyone know of a workaround here so in setups within other test runners'
@playwright/testcontext can be initialised?
I initialize the browser and context in a beforeAll hook with cucumber js. As for settings like timeout they get picked up from playwright.config.js the only issue I’m having is that I need to configure video and trace recording manually in the hooks they are not picked up from the playwright config.
@kumars3 is perfectly right with #11975 (comment). It's not completely necessary to implement a BDD framework for Playwright however hooks into Playwrights' initialisation need to be exposed so that the Playwright context can be initialised. I've done some digging to try and figure out a way and came up with nothing.
Does anyone know of a workaround here so in setups within other test runners'@playwright/testcontext can be initialised?I initialize the browser and context in a beforeAll hook with cucumber js. As for settings like timeout they get picked up from playwright.config.js the only issue I’m having is that I need to configure video and trace recording manually in the hooks they are not picked up from the playwright config.
I'm experiencing the same issue. It also affects the toMatchSnapshot assertion.
@pavelfeldman this is the 3rd most popular issue on the playwright. Is there any workaround you know of to use another test runner by initialising @playwright/test context in a similar manner?
A lot of companies won't consider using Playwright Test without Cucumber because they would be migrating from their current framework which already supports Cucumber and BDD, and it's an important part of their development process so it's a necessity
@pavelfeldman is it possible to get an update on this please? Even if that's to give a rough timeline or just to say it's not currently on the horizon. This will help people, such as myself, who are looking at automation tools to decide whether they can initially work without Cucumber and integrate it later knowing it's coming within a certain timeframe, or that they just need to use a different tool.
I am also in the same situation, not able to ask teams ( I am referring to hundreds of teams basically the whole company I work at) to use Playwright as on another side I was promoting people to use Cucumber BDD frameworks! A combination of both would be awesome! I am certainly not interested to use Tallyb's framework as it's becoming difficult to use it with the latest playwright features! But she certainly did a good job making one for people!
@pavelfeldman @mxschmitt Please let us know if this will be taken up anytime soon?
Is there testing without BDD?
Also experiencing #14539 (Cannot use .toMatchSnapshot() outside of playwright test runner)
I'm in the same boat as many of the earlier posters: We would migrate to playwright instantly if there was "native support" for BDD - as in an adapter for Cucumber resulting in all playwright features being usable in tandem with test cases written in Gherkin.
I have ended up recommending to use WebdriverIO for one of the team who is currently using proctractor cucumber framework. It would have been awesome if playwright test can offer cucumber out of box. May be in future I can recommend some teams to use Playwright Test instead.
It would be cool if BDD with playwright comes handy. I have tried with cucumber - playwright part , yet I personally feel, we would miss the stunning use cases of playwright test runner. Please consider to plugin BDD with playwright.
@pavelfeldman @mxschmitt Please let us know if this is on the roadmap. I have a team of 200 developers and testers who approached me for automation solutions so I was thinking about this.
In the mean time, it exists an acceptable alternative: use the playwright steps to explain all Given/When/Then assertions.
The solution is light and fonctionnal.
Ex:
import { test, expect } from '@playwright/test';
test('Scenario explanation', async ({ page, context }) => {
await test.step('Given ...', async () => {
...
});
await test.step('When user make a stuff', async () => {
...
});
await test.step('Then a stuff is displayed on page', async () => {
...
});
await test.step('And user click on a button', async () => {
...
});
await test.step('Then an another stuff is displayed', async () => {
...
});
});
And the report (with html reporter) is very readable.
In the mean time, it exists an acceptable alternative: use the playwright
stepsto explain all Given/When/Then assertions.The solution is light and fonctionnal.
Ex:
import { test, expect } from '@playwright/test'; test('Scenario explanation', async ({ page, context }) => { await test.step('Given ...', async () => { ... }); await test.step('When user make a stuff', async () => { ... }); await test.step('Then a stuff is displayed on page', async () => { ... }); await test.step('And user click on a button', async () => { ... }); await test.step('Then an another stuff is displayed', async () => { ... }); });And the report (with html reporter) is very readable.
Yes, I am using this currently. Currently this approach is causing me to repeat code and maintenance difficult. Just trying to see if cucumber support would remove repeating same code related to given when then. Any how thanks for inputs.
Up
Up
In the mean time, it exists an acceptable alternative: use the playwright
stepsto explain all Given/When/Then assertions.
The solution is light and fonctionnal.
Ex:import { test, expect } from '@playwright/test'; test('Scenario explanation', async ({ page, context }) => { await test.step('Given ...', async () => { ... }); await test.step('When user make a stuff', async () => { ... }); await test.step('Then a stuff is displayed on page', async () => { ... }); await test.step('And user click on a button', async () => { ... }); await test.step('Then an another stuff is displayed', async () => { ... }); });And the report (with html reporter) is very readable.
Yes, I am using this currently. Currently this approach is causing me to repeat code and maintenance difficult. Just trying to see if cucumber support would remove repeating same code related to given when then. Any how thanks for inputs.
To avoid code repeating on "given/when/then" you can extract the steps like this:
async function checkStep(page: Page) {
await test.step('When ... common step', async () => {
await ...
});
}
and use it in tests:
test('My test which is extracted steps', async ({ page }) => {
await checkStep(page);
});
Plus, if you need, you can even implement the Page Object Pattern
Using test.step() doesn't solve the problem.
Many companies use protractor + cucumber and business wants to see those Gherkins as they were after switching to Playwright.
It may work for completely new projects, but for migrating existing ones... no.
Using test.step() doesn't solve the problem.
Many companies use protractor + cucumber and business wants to see those Gherkins as they were after switching to Playwright.
It may work for completely new projects, but for migrating existing ones... no.
Right now I need for two business units who are on proctractor and selenium with cucumber. So test.step is not solving it. I am trying WebdriverIO but it's not as good as playwright but I am not having choice other than going with WebdriverIO .
Up
is there any way to run cucumber test parallel with playwright
is there any way to run cucumber test parallel with playwright
Cucumber takes an argument --parallel 3 which you can use to run tests in parallel https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md
yes, bdd is good to have for playwright.
The strong point of using cucumber is that you use feature files to explain what is tested. Using test.step and even extracting steps so that they can be re-used is a small step for software engineers to make the test better readable but is still not that what we find in cucumber and using feature files.
Tally's example of how to use cucumber icw playwright works for me (https://github.com/Tallyb/cucumber-playwright) and I don't understand why so many people want to have native support from playwright/test
The suggestion from @mortelee works fine. 👍
However, there are some things that we can't use outside of Playwright runner, like:
.toMatchSnapshot(name[, options])
.toMatchSnapshot([options])
.toHaveScreenshot(name[, options])
.toHaveScreenshot([options])
Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.
The suggestion from @mortelee works fine. 👍
However, there are some things that we can't use outside of Playwright runner, like:
.toMatchSnapshot(name[, options]) .toMatchSnapshot([options]) .toHaveScreenshot(name[, options]) .toHaveScreenshot([options])Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.
To get around the snapshot stuff I created a function
function toMatchSnapshot(name: string, actual: any) {
const file = `${process.cwd()}/snapshots/${name}.json`;
if (!fs.existsSync(file)) {
fs.writeFileSync(file, JSON.stringify(actual, null, 2));
}
const expected = require(file);
expect(actual).toMatchObject(expected);
}
Then('the BEE configuration has rules', function (dataTable: DataTable) {
const [{ id = 1 }] = dataTable?.hashes()?.length === 0 ? [{}] : dataTable.hashes();
const name = id ? `${this.cucumberScenarioName}-${id}-rules` : `${this.cucumberScenarioName}-rules`;
toMatchSnapshot(name, this.beeConfiguration.rules);
});
I am using https://github.com/Tallyb/cucumber-playwright to integrate Playwright and Cucumber. The biggest thing for me that is missing is the integration with the VSCode extension - I liked being able to run/debug tests direct from the test explorer in VSCode, whereas now I must run and debug all my tests via the command line. This isn't a huge difficulty, but it's a shame, and it does make for more friction when working with tests. I'd like to be able to run my BDD/gherkin-based tests from within test explorer as well.
Yes this feature is must , microsoft team please bring a native support playwright + BDD … thanks
I agree with the folks, this feature will help a lot of people where BDD is a requirement.
Currently I'm in the same situation as many folks here, using cucumber-js with a customized world config (a bit overcomplicated due multiple browsers vs devices).
Hope some solution comes up soon.
Excited to be a beta tester for it. I'm currently replacing from an old Java8+Selenium+Cucumber framework to Typescript + Playwright + Cucumber (around 5K test scenarios).
Please, please!
playwright with a native integration with cucumber/BDD/Gherkin it will be extremely helpful !
Hello great PW team, this has over 150 votes, probably the most voted thing , how much more are needed ;)
Adding another vote for this. Strongly second the comment about losing the functionality of the extensions. And it seems that the custom error messaging on expects does not come through, either. If I throw an error, it shows up as expected. But the custom expect message is lost completely.
Please let us know your opinion on this feature request ? I was wondering if this will happen in next 12 months !
I am using https://github.com/Tallyb/cucumber-playwright to integrate Playwright and Cucumber. The biggest thing for me that is missing is the integration with the VSCode extension - I liked being able to run/debug tests direct from the test explorer in VSCode, whereas now I must run and debug all my tests via the command line. This isn't a huge difficulty, but it's a shame, and it does make for more friction when working with tests. I'd like to be able to run my BDD/gherkin-based tests from within test explorer as well.
@yaffol My demo repo contains a way to run the tests directly from vscode, by using the debugger: https://github.com/Tallyb/cucumber-playwright#in-visual-studio-code
(for the record: I also upvoted this feature, but until we have it, you are welcome to use my repo).
The suggestion from @mortelee works fine. 👍
However, there are some things that we can't use outside of Playwright runner, like:.toMatchSnapshot(name[, options]) .toMatchSnapshot([options]) .toHaveScreenshot(name[, options]) .toHaveScreenshot([options])Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.
To get around the snapshot stuff I created a function
function toMatchSnapshot(name: string, actual: any) { const file = `${process.cwd()}/snapshots/${name}.json`; if (!fs.existsSync(file)) { fs.writeFileSync(file, JSON.stringify(actual, null, 2)); } const expected = require(file); expect(actual).toMatchObject(expected); }Then('the BEE configuration has rules', function (dataTable: DataTable) { const [{ id = 1 }] = dataTable?.hashes()?.length === 0 ? [{}] : dataTable.hashes(); const name = id ? `${this.cucumberScenarioName}-${id}-rules` : `${this.cucumberScenarioName}-rules`; toMatchSnapshot(name, this.beeConfiguration.rules); });
@benjefferies Did you use this for image (screenshot) comparison?
Could you give some more context on this? e.g. what is in beeConfiguration.rules and why are you loading .json file?
thanks
@csabasolyom I haven't use it for image comparison. All I've done is create a custom snapshot assertion check similar to https://jestjs.io/docs/snapshot-testing
Built-in support for BDD would be great. Pure frontend tests can do without, but for real end-to-end tests on more complex applications, you need your tests to speak the language of the domain, and you need more layers in your test automation architecture. Something like specs/feature files (tests), step definitions, page objects and API clients. It doesn't necessarily have to be a DSL like Cucumber though. But some guidance and support in the framework how to build such test automation architecture with Playwright would be welcome.
This would be great to have! I'm also using another implementation, but BDD is important for scalability.
I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.
I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.
I love Playwright but I really can't understand why the devs won't say anything about their most commented issue.
Hi @pavelfeldman how are you doing?
Do you have any update regarding this feature request?
When comparing solutions for E2E testing I found that webdriverIO & cypress support cucumber through the packages:
@badeball/cypress-cucumber-preprocessor
@wdio/cucumber-framework
In playwright it's possible using cucumber-js but you'll miss all the advantages of the playwright test runner.
So an implementation within playwright increase the change ofus migrating to it :)
I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.
I love Playwright but I really can't understand why the devs won't say anything about their most commented issue.
This could be complex and complicated to implement otherwise they have would done it already as its been a year now since this request was originally logged.
I really love playwright, but sadly because of this feature not being implemented, I can't offer this as a Test Automation solution for my company.
Please add this feature.
The handling of the page object from the session is far from perfect and also adding new URLs to the pytest objects as property should be changed (don't know how to do it better), but this is the only option for now, if you want to have BDD, Data Driven testing, Playwright and Allure reporting. I know there is much work to do (dead code, IDE integration). But my resources are limited during my free time:
https://github.com/MartinB134/playwright-pytest
I will appreciate any Feedback and participation!
Do you guys @pavelfeldman @aslushnikov have any update for this request? This has been requested a year for now and I see there are so many people upvoted for this so.
Any udpate on this. Eagerly waiting.
While waiting official comment I'd suggest to discuss, what actually do you expect here. I see several options:
- Using playwright test-runner that reads and executes BDD tests written in gherkin
- Using cucumber test-runner that have easy way to integrate playwright in steps
- Using
Yes we can do that as well. I would prefer playwright test runner if we can make it in anyways 😊
I think we are waiting for the first option -> 1. Using playwright test-runner that reads and executes BDD tests written in gherkin
Hello,
Ideally this would be also backwards compatible with current tests implementations, meaning steps or assertions could be marked as a BDD step.
Use case:
We have already added cucumber-js within our test suite cause it was a business requirement, and we are using same test base with cucumber and playwright runner.
Cucumber is used for business people, playwright for everything else.
I would hate to need to rewrite everything that we have currently written to fit it in, would probably stick to current implementation in that case.
Example
Base function
export const clickAgreeWithDisclaimer = async ({ page }: TestArgs): Promise<void> => {
await page.getByTestId(AGREE_BUTTON).click();
};
Cucumber feature file
Background:
Given I agree with disclaimer
BDD step
Given('I agree with disclaimer', () => agreeWithDisclaimer({ page, context }));
Playwright
test.describe('Feature: Overview', () => {
test.beforeEach(async ({ page, context }) => {
await agreeWithDisclaimer({ page, context });
});
});
If I was to describe my ideal developer experience in terms of using the tools, it would be to be able to use playwright's vs-code extension to see and run my tests, with the great tooling like tracing, but to be able to run tests derived from BDD features there.
And also to be able to run them via the CLI, but that goes without saying.
I guess what I am trying to say is that I really appreciate the DX (Testing eXperience?) of playwright/test, the trace viewer and now UI ('watch') mode, and would really like to be able to use them **as well as ** tests that are executable specifications.
Hey guys @dgluzman, @pavelfeldman, @aslushnikov, have you ever discussed that feature in your team meetings, please please share some update on it, many people are wondering if there is a chance to see it.