Lighthouse running against the live theme instead of the generated development/preview theme

[darrylb-github]

Hey,

Not sure if I'm missing something, but when trying to use this it seems to generate a preview theme successfully, but then not run lighthouse against it, instead running lighthouse against the live theme from what I can tell.

My workflow:

name: Lighthouse
on: [push]
jobs:
  lighthouse:
    name: Lighthouse
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Lighthouse
        uses: shopify/lighthouse-ci-action@1.0
        with:
          store: '***.myshopify.com'
          
          app_id: ${{ secrets.SHOP_APP_ID }}
          app_password: ${{ secrets.SHOP_APP_PASSWORD }}
          lhci_github_token: ${{ secrets.LHCI_GITHUB_TOKEN }}

Logs with store name redacted:

Creating development theme
{"theme":{"id":125912940720,"name":"Development (12a77e-3d8c52211dfc)","role":"development","shop":"***.myshopify.com","editor_url":"https://***.myshopify.com/admin/themes/125912940720/editor","preview_url":"https://***.myshopify.com/?preview_theme_id=125912940720"}}
==============================
Configuring Lighthouse CI
==============================
Running Lighthouse CI
✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
⚠️   GitHub token not set
Healthcheck passed!

Getting a new page...
Getting preview cookie...
Running Lighthouse 3 time(s) on https://***.myshopify.com/?_fd=0&pb=0

If I load the preview_url I get the expected development theme based on my current branch, but the lighthouse run for my branch runs against the normal store URL without the preview_theme_id set, and when loading up the report screenshots at the end of lighthouse I can see it's my live theme and not the development theme I'd expect for the branch lighthouse is being run against and generating a preview URL for.

Am I doing something wrong?

Thanks,
Darryl

[darrylb-github]

Looking more at the code I see it uses the preview url via a cookie, but still not sure why it's running against the live theme instead of the preview url 🤔

[abmenzel]

+1 I am having the same issue. I am running this on a clone of the dawn theme, but the lighthouse action is running on the live theme that we are comparing the new theme against.

[christiandp7]

+1 I have the same issue. It;s running the live theme instead of the development theme.
Creating development theme ERROR update theme/layout/theme.liquid: 404 ERROR update theme/sections/cart-sidebar.liquid: 404 ERROR update theme/sections/cart.liquid: 404 ERROR update theme/sections/header.liquid: 404 ERROR update theme/sections/footer.liquid: 404 ...

[charlespwd]

Ugh. Looks like a Shopify CLI bug.

Could anyone hardcode the store in the GitHub workflow file and paste the entire logs in here?

[lukechadwick]

Any movement on this? I seem to be seeing the issue on our store, preview URL seems to be correct but the scores/report screenshots are of our live store.

[muriloshimizu]

Any updates on this? Thanks!