executeautomation/mcp-playwright

Code generation uses quotes that result in syntax errors

Opened this issue · 0 comments

I'm not sure if this is an error in the LLM that can be fixed via tuning the prompts in the code generation part of your utility. Example output code:

import { test } from '@playwright/test';
import { expect } from '@playwright/test';

test('DriveThruRPG_2025-04-08', async ({ page, context }) => {
  
    // Navigate to URL
    await page.goto('https://www.drivethrurpg.com');

    // Click element
    await page.click('[data-section-cy='bestselling'] a[href*='/product/']');

    // Click element
    await page.click('[data-cy='addToCartProductButton']');
});

Note that it's using single quotes for everything, resulting in syntax errors in the Playwright code.

As an unrelated note, which is probably more LLM related and perhaps harder to fix: tests were missing. I will list my instructions below, and even though the text feedback said that the generated code included a test to verify that the item was in the cart, you can see that there was no code actually verifying this.

Open the browser
Go to https://www.drivethrurpg.com
Click on the first link in the Bestselling Titles section that goes to a page with /product/ in the path. This section is identified by the HTML attribute data-section-cy="bestselling"
Wait for the page to load
Remember the title of the product, and add that item to your cart using the button identified by data-cy="addToCartProductButton"
Verify that particular product is in your cart, by checking all elements with the HTML attribute data-section-cy that start with "cartProductRow" and see if the text of the item's title is present.
Close the browser