Problem reinstalling, but no error logs

Question

Problem reinstalling, but no error logs

bilogic opened this issue 10 months ago · 43 comments

For bug reporting only! If you're posting a feature request or discussion, please ignore.

Expected Behavior

The (re)installation should proceed smoothly

Current Behavior

App was installed on this shop before. It was uninstalled and now trying to re-install again.
But the reinstalling keeps failing and returning to the install screen.

Failure Information

There are no error logs in laravel.log

Steps to Reproduce

~~Unknown, it has happened more than once and can be quite hard to reproduce~~

If users.deleted_at remains as null for whatever reasons when the app is uninstalled, this problem will surface
deleted_at only gets a value when Shopify's uninstall webhook succeeds, but webhooks can fail.

Disable all webhooks in the shopify app (to simulate that Shopify has stopped retrying and all webhooks have failed)
Install the app
Uninstall the app (with webhooks disabled, users.deleted_at remains as null)
Reinstall the app - unable to, leads to 404 on shopify backend

Hope this can be addressed.

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

Package Version: v19.0.0
Laravel Version: v8.83.27
PHP Version: v8.1.27
Template Engine: Blade
Using a toolset (Docker, Laradock, Vagrant, etc.): n/a

Investigation to find a solution

Triggered javascript breakpoints

/authenticate <= auth/full.blade.php
click INSTALL
/token <= layouts/default.blade.php, before createApp
/token?embedded=1 <= layouts/default.blade.php, before createApp
/token?embedded=1 <= token_handler.blade.php
/billing <= billing/full.blade.php
click APPROVE
/token <= layouts/default.blade.php, before createApp
/token?embedded=1 <= layouts/default.blade.php, before createApp
/token?embedded=1 <= token_handler.blade.php.php
/ <= layouts/default.blade.php, before createApp
click UNINSTALL
click INSTALL
/token <= layouts/default.blade.php, before createApp
404

Answer 1 · 2024-03-13T15:15:55.000Z

I have to delete the user in order to break the install cycle, this package should have a way to detect and automatically break the cycle.

Answer 2 · 2024-03-14T06:52:34.000Z

In the VerifyShopify middleware

if ($tokenSource === null) {
    // Check if there is a store record in the database
    if ($this->checkPreviousInstallation($request)) {
        // Shop exists, token not available, we need to get one
        $a = $this->handleMissingToken($request); // this line causes the install loop
    } else {
        // Shop does not exist
        $a = $this->handleInvalidShop($request);
    }

    return $a;
}

Answer 3 · 2024-03-15T01:14:52.000Z

It seems that we can also check for 401/403 failures under 2. Failing API requests https://www.littlestreamsoftware.com/labs/what-to-do-when-a-shopify-app-is-uninstalled/

I suppose this check should be part of checkPreviousInstallation so that handleInvalidShop gets called if we have been uninstalled but not properly informed.

Answer 4 · 2024-03-19T18:17:41.000Z

@bilogic do you have the uninstall job set up

Answer 5 · 2024-03-20T01:54:07.000Z

I do have an uninstall job set up, but I don't think that is the issue.

Webhooks can and do fail. How are we handling that today?

Answer 6 · 2024-03-20T10:28:14.000Z

@bilogic the package does not handle failures automatically, you'd have to build this into the jobs you want to use from each webhook.

Answer 7 · 2024-03-20T10:55:12.000Z

@Kyon147

It is not about what the webhook does.

What if the network/server goes down for extended periods and the webhook does not reach our server?

Answer 8 · 2024-03-20T11:12:18.000Z

If your server goes down then Shopify we try and resend the webhooks a number of times before it stops. It's all in their docs somewhere how often and how many tries they do.

Answer 9 · 2024-03-20T13:58:30.000Z

Yes. And when it stops and all attempts failed, it causes the infinite install loop, effectively permanently preventing this store from ever installing my app again.

I'm not sure why you see this is a non issue or why this package should not be responsible for this problem.

Answer 10 · 2024-03-20T14:43:42.000Z

Hi @bilogic

If the webhooks fail and we don't set the deleted at event and this is causing an error for the re-install then we need to refactor that re-install logic and use other fields to determine it potentially.

I'd be happy for you to open a PR if you wanted to submit something that could solve this edge case.

If not, I can take a look when I have some time.

Answer 11 · 2024-03-20T14:44:27.000Z

Ok, thanks, now we are on the same page. I will look into it and attempt a PR.

Answer 12 · 2024-03-20T14:46:11.000Z

Btw, I'm not that familiar with the Shopify API, which was why I relied on this package in the first place.

So offhand, do you have any opinions on how this can be addressed? e.g. which Shopify API should I attempt to try to determine if my app has been uninstalled etc?

Answer 13 · 2024-03-20T14:48:44.000Z

If the store uninstalled the app, then you would not be able to use any API anyway.

We'd need to solve this using data we have in the users table at the time of install, that is the only data points we have when we don't have an api key to use.

Plus, we don't want to slow down the install by making requests externally really.

Answer 14 · 2024-03-20T14:52:47.000Z

Ok, noted on not making any external requests.

I did use an interactive debugger to try and understand the install lifecycle of this package, but I wasn't able to make much sense of it. Do you know of any references that is clear in explaining the installation, preferrable in a sequence diagram?

Or if you have an idea on how we can figure things out, I would be more than happy to just write and test the implementation.

Answer 15 · 2024-03-20T17:00:59.000Z

There's no documented flow outside of what is already in the wiki which is a high level overview.

For the install, it's mainly in InstallShop https://github.com/Kyon147/laravel-shopify/blob/master/src/Actions/InstallShop.php

There's likely an error in the restore method on that file which is breaking if deleted_at is null but this is a educated guess as I'd need to go through the flow myself and debug it to be sure.

Answer 16 · 2024-03-20T17:02:56.000Z

Alright, let me check out InstallShop and update if I find anything interesting. Thanks!

Answer 17 · 2024-03-21T01:01:00.000Z

Installation flow

https://shopify.dev/docs/apps/auth#authorization

Auth

Managed (recommended)
- https://shopify.dev/docs/apps/auth/installation
Auth code grant
- https://shopify.dev/docs/apps/auth/get-access-tokens/authorization-code-grant/getting-started

It would appear that this package is performing utilizing 2

Token acquisition

Token exchange (recommended)
- https://shopify.dev/docs/apps/auth/get-access-tokens/token-exchange
Auth code grant
- https://shopify.dev/docs/apps/auth/get-access-tokens/authorization-code-grant

Unable to tell which one this package is using yet

Answer 18 · 2024-03-21T05:45:59.000Z

Updated the OP with a bunch of triggered javascript debugger breakpoints

Answer 19 · 2024-03-27T15:47:50.000Z

Just to update, I'm looking into this, this week.

Answer 20 · 2024-03-27T16:29:47.000Z

Wonderful, thank you so much. Please ask if you have trouble reproducing the problem.

Answer 21 · 2024-04-03T00:19:11.000Z

Curious to know if there is progress on this. Thank you.

Answer 22 · 2024-04-03T08:51:37.000Z

@bilogic not got to it just yet as I worked on testing the laravel 11 PR - will try and get to it this week

Answer 23 · 2024-04-13T18:08:13.000Z

Hello @Kyon147 @bilogic

I'm encountering a similar issue and seeking an update. Currently, I'm facing two specific problems:

AppUninstalledJob isn't being triggered: Despite expectations, AppUninstalledJob fails to execute as intended.
After_authenticate_job issue: There's a similar hiccup with after_authenticate_job. After users uninstall and then reinstall the app, the job doesn't initiate. This issue has been experienced by approximately 10-15 users in the LIVE application recently.

=> To give you a clearer picture, here's how I've set up AfterAuthenticateJob:

'after_authenticate_job' => [
          [
            'job' => \App\Jobs\AfterAuthenticateJob::class,
            'inline' => true
        ],
    ],

Additional Information:

Package Version: 19.0.0
Laravel Version: 10.10.0
PHP Version: 8.1.0
Template Engine: Blade

I would greatly appreciate any assistance you can provide on this matter. Thank you in advance for your help! 🙂

Answer 24 · 2024-04-14T01:59:19.000Z

@thesunilyadav

I faced your same issue too, but that issue isn't really a problem until it causes this problem that we are now trying to address.

I would suggest opening another issue in order for us to stay focused on this: how does this package self rectify an AppUninstalledJob after all retries have exhausted and still failed.

Answer 25 · 2024-04-15T10:18:44.000Z

As @bilogic suggest can you open a separate issue, I've never seen the after auth fail for my apps personally but it is possible if the uninstall did not happen cleanly.

Hoping I can get the PR sorted asap.

Answer 26 · 2024-04-15T16:24:51.000Z

Hello @Kyon147 @bilogic, I've created a new issue. Please feel free to take a look when you have a moment #Issue_298

Answer 27 · 2024-04-22T01:16:00.000Z

@Kyon147 just curious if you managed to make any progress? Thank you!

Answer 28 · 2024-04-22T09:35:44.000Z

Hey @bilogic

I think I have a working version on my local machine, apologies for the radio silence...partner is about to give birth so juggling things at the mo!

Answer 29 · 2024-04-22T09:45:17.000Z

ah congrats! if you could, perhaps push a branch let me have a peek? ;) but don't let me inconvenience you thanks!

Answer 30 · 2024-04-28T06:19:33.000Z

@Kyon147 possible to let me have a look? Thank you!

Answer 31 · 2024-05-07T00:07:18.000Z

@Kyon147 hope everything is fine, I would like to help by having a test/look at your solution. Thank you!

Answer 32 · 2024-05-13T02:55:59.000Z

@Kyon147 would love to help if you can share whatever you have at the moment. Thanks!

Answer 33 · 2024-05-17T13:23:57.000Z

Hey @bilogic not forgotten about you or this ticket, just trying to decide on some bits for this before I submit a PR.

As the issue is based on the webhook, we need an "offline" solve for when a store is trashed but the package is not aware the store has been.

Answer 34 · 2024-05-17T13:46:06.000Z

As the issue is based on the webhook, we need an "offline" solve for when a store is trashed but the package is not aware the store has been.

@Kyon147 thanks, well this set of criteria has gotten me stump.

Perhaps there is something in shopify's redirect that we could pick up on? Otherwise it is almost like being able to read shopify's database while being offline.

I did try earlier this year, but found nothing. What do you have already? Perhaps put it on a branch? Maybe we can build on each other's ideas.

Answer 35 · 2024-05-18T22:49:31.000Z

The main issue is that once the app is uninstalled, if we miss the webhook, then that's it.

So my test at the moment is to trying and consistently, re-auth if for some reason we don't have a working api key, but a deleted_at column is still null and assume that we have a "dirty" store install and should treat it as a new install.

Answer 36 · 2024-05-20T07:31:56.000Z

@Kyon147 sounds quite similar to what I had in mind previously.

But I wasn't able to determine if the API key I have is still working. Is this re-auth performed on every interaction with our UI? Or only during the installation request?

Answer 37 · 2024-05-26T12:54:31.000Z

@Kyon147 would be great if I can see and build on what you already have thanks

Answer 38 · 2024-06-03T04:31:25.000Z

@Kyon147 just checking in with you

Answer 39 · 2024-06-15T09:50:35.000Z

Hey @bilogic

Apologies, new dad life, work and many other projects just sucking up all my time.

I'll try and get a PR out this weekend for you 👍

Thanks for being patient with me

Answer 40 · 2024-07-31T08:07:21.000Z

Any news ?? Thanks !