App permissions deleted after reinstall?
Closed this issue · 35 comments
Hello,
I understand you dont recommend dirty flash etc. Still reinstalling the package is practically unavoidable :P
But even a simple case of update (from previous april version to the latest one) causes this problem. It also caused for me in the past on two different devices if I needed to reinstall MinMicroG.
Both such devices are on Lineage 17.1, one is normal i.e. proper, another -- A/B. A/B btw is very annoying, and each time I upgrade I have to reflash all zips including magisk and your MinMicroG, which again generates the problem.
Basically reflash screws over all app permissions, including those nothing to do with location. And I mean all apps, even those which never had location enabled in the first place. E.g. browsers lose storage access permissions, camera apps lose access to camera, audio etc. And even location permissions are not setup correctly then. I.e. in this case I have to allow manually location permission for all location backends in the microg.
Environment:
two different devices both running on latest LineageOS 17.1
Magisk 20.4
MinMicroG various versions, using NoGoolag package. Installed with default options, i.e. as a magisk module.
No gapps, obviously.
Do you flash the zip from Magisk Manager or recovery?
If from Manager, then I'm afraid nothing can be done about it while on a running system. It's the reason NanoDroid has completely disallowed flashing from Magisk Manager.
A dirty flash through recovery would wipe permissions for your user apps but should preserve permissions for your system apps. If this happens for you when you flash from recovery, then that's pretty weird. Could you get me the logs from the recovery if so?
-
Always from TWRP.
-
Just couple of days ago updated my A/B op7p device. That is the standard proper flashing (the other options would create problems):
- Boot into TWRP active slot
- Install LOS new build which always go to inactive slot
- Install TWRP (which goes both slots) and then reboot (to previously inactive slot)
- Install Magisk, install Bromite magisk module, install yours, install remove force-encryption (always need to be last flash as per instructions from the developer)
- Reboot system
Unlike a few weeks ago when doing all things properly everything went smooth with LOS. Booted updated LOS first time with magisk etc enabled. But permissions are screwed. E.g. all storage permissions from user apps got revoked. GCam, normal LOS camera etc all lost their permissions etc.
Also worth noting -- after the process above -- all my previously selected location providers become unselected. Many (e.g. GSM location, Wifi location service) did not have their location permission enabled.
After enabling all manually from settings menu and reboot all appears to work, including microg and location services.
Well firstly, if you haven't been able to restore all permissions for system apps yet, you can go to the recovery and delete /data/system/users/0/runtime-permissions.xml by hand. It will wipe all permissions for user apps, but it will fix all system perms to granted.
About debugging why this happened, I'm afraid the current information atleast without a log is too vague to figure something out from. So instead I'll just tell you all the technicals of what happens with the permissions, maybe it'll help.
Android doesn't really like when certain kinds of system apps are added spontaneously after the first boot. The runtime permissions database is stored in /data/system/users/*/runtime-permissions.xml (that is created on first boot), and it gets corrupted when Android gets confused after something like that happens. System apps and user apps both lose permissions randomly.
So that's why when installing those apps as system (not just microG packs like MinMicroG or NanoDroid, GApps packages do it too), you have to delete that database so Android can create a fresh one with these new system apps included. All permissions for user apps will be reset, but atleast system apps won't go bonkers.
But, of course, there are a couple of condition in which the database is not deleted (one of which your device was possibly in):
- If flashing from Magisk Manager (because deleting system file while Android is running is not a pretty idea)
- If the data partition is encrypted or for some other reason
- Something else went awry?!
-
I dont think that system apps have problems. Actually I've just checked (by enabling systems apps in the menu first) -- for instance bluetooth system app has all the permissions requested, even though I dont use bluetooth so didn't fix its permissions manually.
-
Hold on. So what are you describing is exactly what happens on any of my phones, right? Apparently the permission database gets deleted? As all user apps lose their permissions? So it is deliberate?
-
What kind of logs you need? TWRP install logs or what?
OK new development.
avoiding A/B now, and even dirty flash. On non a/b device which is op3 I'm doing the following:
reboot to TWRP
Flash your latest zip, as no rom is flashed, no magisk, or any other zip.
Do a little check <3
Reboot.
All user apps permissions are gone.
But thats actually not surprising now -- as per little check before. The file exists before but after flash and before reboot the file is gone. And the file name is /data/system/users/0/runtime-permissions.xml :LOL:
Basically it seems like an annoying manual workaround for now (if not reflashing rom) is to copy that file manually to /sdcard and then copy it from TWRP after flash but before reboot. All permissions as before.
Basically your zip when flashed from TWRP basically wipes permission file.
Ohhh I see. My bad, I actually interpreted your comment to mean that system permissions were lost too, which would have been a bigger problem.
So it is deliberate?
Yep, it is intentional (code here). If we don't do that, then usually we will face a much worse problem (system permissions being lost too).
Basically it seems like an annoying manual workaround for now (if not reflashing rom) is to copy that file manually to /sdcard and then copy it from TWRP after flash but before reboot. All permissions as before.
Did you try that workaround? I'm fairly sure you will encounter the system perm corruption problem if you do :P
In case it works, then great! That means that we can write extra logic that does not wipe the permissions database if there is a possibility that user permissions can be preserved under certain conditions.
Yes I have tried that.
Been a few days and a couple of system restarts (manually). Seems to work just fine.
I undestand concerns but still -- wiping all user app permissions is very annoying if someone installs weekly LOS builds.
Nornally I do about once per two weeks, and thats on two devices. That would be four times per month.
You're right, it is pretty annoying, but still, less annoying than losing even system perms.
I'll have to do some experiments to see if I can grep for something in it or something for a conditional delete in case the perms won't be destroyed.
In the meanwhile, you can install the zip from Magisk Manager through which perms won't be deleted in any case.
In the meanwhile, you can install the zip from Magisk Manager through which perms won't be deleted in any case.
Interesting. I wasn't aware about that. Maybe I can create a sure method of installation based on this.
Again, the real problem is not normal proper devices, like my second device. As this currently basically requires to install from twrp. Problem is that they (android devlopers) fixed what was not only not broken, but working great.
On normal devices any update from built-in updater in LOS reboots device into recovery (whatever is actually installed) and then runs all the required scripts. Including magisk persistent installation. It is a completely different story on A/B devices.
Basically LOS developers should enable the old install method for advanced users (like in developers options) and that would solve a lot of problems, including this one.
@pwn0r it took a while without any progress from my side :P
But I only recently got to know that the addon.d mechanism (responsible for preserving mods across ROM updates, like Magisk or OpenGApps) in MinMicroG was entirely broken on A/B devices. The last release seems to fix that for the few A/B owners I talked to.
Just to confirm that I understand correctly, this problem stems from your ROM deliberately not respecting addon.d, right? And this release did not fix ('avoid' would be more apt, I suppose) the issue altogether here?
Well, I normally update roughly every two weeks. LOS has weekly updates now and sometimes even misses those updates. I've just tried an update. Before the update I had: Magisk 21.1, Lineage 20201122, MinMicroG nogoolag 2.8.
Installed 20201129 as normal (i.e. twrp installation, not automatic update). Only now upgraded MinMicroG from 2.8 to 2.9. User application permissions were lost after I rebooted into LOS.
As for addon.d I'm not sure. First of all LOS on this device is to be updated as a normal A/B device, not using recovery at all. Yet the same version of LOS on non-AB device can be updated normally, and magisk automatically restores itself in twrp. This would mean that addon.d works in the latest LOS???
In fact, when I install LOS from TWRP on oneplus7pro (which is obv A/B device), magisk installation runs automatically after LOS install. Just that I dont trust that anyway on the A/B device and flash it manually again.
Just to remind my install sequence -- on A/B devices only -- reboot into TWRP recovery active slot, install LOS from file, which installs to inactive (and kills TWRP there), after that magisk install runs automatically. Then I run TWRP install (which installs to both slots), reboot into recovery again. Now the 2nd slot is active, then I flash magisk, flash minmicrog and disabledmverity/encryption. After that reboot into LOS.
Yeah, if Magisk has started restoring itself after every ROM update, it should just work out without you having to flash everything again. You should try that once and see what happens.
I think Magisk clears the entire module storage directory when you reflash it as a zip manually (but of course, keeps modules intact when restoring itself automatically on updates). So what's essentially happening here is that flashing Magisk clears all traces of MinMicroG having ever been installed, and when you flash MinMicroG again, it has no way to know that it was already installed before, and thus sees wiping permissions as a required action.
finally something done :P Well the problem was at least partially due to magisk (21 and 21.1 version I believe). They changed addond there into v2 (dont ask me whats the difference, apparently it is hard to find a proper documentation on addond) for LOS17.1 on A/B devices, which actually broke it for non-AB devices from TWRP.
basically if you did a dirty flash from twrp the addon.d script was never working for magisk and therefore you had to manually reflash it. Which apparently not good for permissions on minmicrog, right?
The PR which fixed the issue is topjohnwu/Magisk#3494
which was included into 21.2 magisk version. Basically now at least the permissions seem to work properly on non-A/B devices when you reflash/update LOS from twrp.
Ahhhh. That makes sense.
Now that Magisk preserves itself, you won't need to reflash minmicroG, and even when you do update, it won't wipe your permissions.
... Point of order; you might still have to reflash magisk, because the TWRP installer might still clobber the Magisk patches to the initramfs ramdisk. Order of operations:
- boot into TWRP recovery
- flash Lineage update zip (which installs to inactive slot), which automatically triggers addon.d survival script to execute, so far so good - it automagically installs magisk to inactive slot on top of the OTA you just installed. Flag is set for next boot to be from inactive slot. This is where, hopefully, MinMicroG will preserve itself if you've done a system-mode install.
- flash TWRP installer zip because Lineage doesn't include it (patches recovery ramdisk in both slots), which will nuke Magisk anyway because they're both inserting themselves into the same initramfs ramdisk and DOESN'T trigger the addon.d survival script again -- really, they need to fix that installer script so it plays nice with Magisk and addon.d -- But the modules directory should still be there with everything in it, including MinMicroG if you installed it as a module.
- reboot to recovery (you're now using the other slot which has its own system & initramfs ramdisks for boot/recovery/etc) which should be the TWRP you had to manually install in step 3.
- flash magisk again to correctly re-patch the initramfs on the active slot to insert itself (somehow the Magisk installer is able to not destroy TWRP when inserting itself into the ramdisk).
At no point so far have we actually wiped data, or rebooted into system, which is the point; we're just trying to do a small OTA update to the system without having to go through first boot & initial setup all over again, and keep some things intact that keep assuming we're trying to do a fresh install and prevent them from clobbering everything to make a clean slate that we DON'T WANT. There should be no system permission corruption woes awaiting us in this process, because we're not putting any GApps back in that MinMicroG would have to remove again.
-
Now is the point where one would flash MinMicroG (or any other GApps package) again, in the hopes that it will simply put the same files in the same directories as it did before, whether as module or system install, and not touch the runtime permissions, because they were already created AFTER a clean first boot WITH microg installed -- that is, we're not changing which system apps are installed other than trying to make this /system slot congruent to the old one we just switched away from due to the way A/B devices do their OTA updates. If we could just get the installer to put the things where they go, and print a message saying "if you're not dirty-flashing an OTA update, you should really wipe data now or risk destroying everything you've ever loved" to the console instead of trying to outsmart people, that would be great.
-
Reboot system, and hopefully (if MinMicroG will just put the apks where they go and not wipe permissions) we won't have to reset anything because as far as the system is aware, there's still the same GMS, GSF, Phonesky, sync providers, and swipe library plugins right where they were before all the OTA update and slot switching foolishness.
There won't be many new non-A/B devices because Google is Google and writes the requirements, so the sooner we get used to doing things in a way that plays nicely with that brain-damaged setup they're forcing on us, the better.
Alright, so @Terminator-J and I talked, and there are several other problems with the A/B workflow too.
So I'm going to change the 'dirty-flash detection system' to just store its mark inside the data partition (where the actual things it cares about reside), so that it is not wiped with either magisk reflashes or slot changes. Honestly I should have thought of this sooner and then we wouldn't have had to spend this long looking for problems in other places, that's on me. I believe that this solution will fix both of your current problems.
Also, it seems that the addon.d is not working with a new format required by custom ROMs. That will have to be looked into and updated.
Alright, did the thing with 052934d. Future flashes should not wipe your permissions unneccesarily after this.
Also added the tag to make addon.d v2 work, but I have yet to add the A/B slot detection mechanism so it still doesn't work when switching slots.
Here's the CI builds:
https://github.com/FriendlyNeighborhoodShane/MinMicroG-abuse-CI/releases/tag/2020.12.31
... Point of order; you might still have to reflash magisk, because the TWRP installer might still clobber the Magisk patches to the initramfs ramdisk. Order of operations:
- boot into TWRP recovery
- flash Lineage update zip (which installs to inactive slot), which automatically triggers addon.d survival script to execute, so far so good - it automagically installs magisk to inactive slot on top of the OTA you just installed. Flag is set for next boot to be from inactive slot. This is where, hopefully, MinMicroG will preserve itself if you've done a system-mode install.
- flash TWRP installer zip because Lineage doesn't include it (patches recovery ramdisk in both slots), which will nuke Magisk anyway because they're both inserting themselves into the same initramfs ramdisk and DOESN'T trigger the addon.d survival script again -- really, they need to fix that installer script so it plays nice with Magisk and addon.d -- But the modules directory should still be there with everything in it, including MinMicroG if you installed it as a module.
- reboot to recovery (you're now using the other slot which has its own system & initramfs ramdisks for boot/recovery/etc) which should be the TWRP you had to manually install in step 3.
- flash magisk again to correctly re-patch the initramfs on the active slot to insert itself (somehow the Magisk installer is able to not destroy TWRP when inserting itself into the ramdisk).
This is exactly the sequence I do as per earlier posts. Had to find it the hard way after corrupting at least one slot (not bootable) once or twice when I purchased one7p. If you want both magisk and twrp working rebooting to TWRP twice is unavoidable. If you reboot to recovery only once, you sacrifice twrp, and thats not an option. Bult-in LOS recovery is such a monstrosity that needs to be killed with fire -- it does not have an option to change an active boot slot, for starters.
One other point to add -- people also may use a force decrypt, which is a very good thing actually. However with a decryptor you need to reboot again to recovery again anyway (and it must be TWRP as LOS recovery is useless). If you force-decrypt your phone you have to enforce it every time you flash a LOS update. Otherwise you are guaranteed to soft-brick your phone (both slots become unbootable and you need to format data and recover from backup to make it work again).
- flash TWRP installer zip because Lineage doesn't include it (patches recovery ramdisk in both slots), which will nuke Magisk anyway because they're both inserting themselves into the same initramfs ramdisk and DOESN'T trigger the addon.d survival script again
Finally! Thanks for providing an in-depth explanation as this is actually the reason for this problem.
At no point so far have we actually wiped data, or rebooted into system, which is the point; we're just trying to do a small OTA update to the system without having to go through first boot & initial setup all over again, and keep some things intact that keep assuming we're trying to do a fresh install and prevent them from clobbering everything to make a clean slate that we DON'T WANT. There should be no system permission corruption woes awaiting us in this process, because we're not putting any GApps back in that MinMicroG would have to remove again.
Just a bit of correction here -- system-level permissions are not a problem. User app level permissions are being wiped. Pretty much all of them, including storage/camera etc.
There won't be many new non-A/B devices because Google is Google and writes the requirements, so the sooner we get used to doing things in a way that plays nicely with that brain-damaged setup they're forcing on us, the better.
IIRC A/B de-facto becomes a requirement starting from Android11. That is any device which launches with android11 should have A/B. Thing they keep changing things every major version release. Before it was A/B (starting from android8 but it was not mandatory, though most vendors switched over to A/B by android10), now they have something called virtual A/B on android11.
So I finally got around to it and tried some stuff for addon.d v2. It works on my devices, but I don't have an A/B, as you know.
Here's a test build:
https://github.com/FriendlyNeighborhoodShane/MinMicroG-abuse-CI/releases/tag/2021.01.17
So I finally got around to it and tried some stuff for addon.d v2. It works on my devices, but I don't have an A/B, as you know.
Here's a test build:
https://github.com/FriendlyNeighborhoodShane/MinMicroG-abuse-CI/releases/tag/2021.01.17
seems to work, i have tried it on my one7p. Note that I also use current versions of both twrp and magisk, which are 3.5.0 and 21.4 respectively. I installed updated magisk first, then new microg (which wipes permissions as usual) then reboot, then again to recovery.
So then the normal update scenario seemed to work == install new LOS == install twrp installer == reboot to twrp new slot == reflash magisk == flash encryption removal == reboot to system.
It seems as long as you avoid actually reflashing microg in twrp, all permissions are preserved and microg still works (i.e. provides location via configured location providers).
Dang it, it's still wiping your permissions? I thought I finally extinguished that problem three weeks ago!
Do you see a /data/.mmg file in your recovery? Do you see anything about PREPPER in the recovery logs when you flash the zip?
Anyway, I guess since you have it as a Magisk module, that you don't need to reflash it anymore means Magisk's addon.d is working properly now?
Yet to be seen if minmicrog's new addon.d works on A/B.
Alright, so addon.d-v2 support for A/B devices has finally been sorted out with 4e4f05e, and it seems to work for @Terminator-J too.
Curiously enough, it also didn't wipe permissions for them after the changes made, so something interesting might be going on in your case @pwn0r
yeah, but how do I get this build then? The previous one was from 17 Jan.
Also, can you confirm regarding "not wiping": is it supposed to be not wiping when upgrading the ROM, upgrading MinMicroG or both?
Just in case -- upgrading minmicrog means flashing a new minmicrog zip from twrp, unless there is another way??
Oh my bad mate, I forgot to trigger a CI build for that last change, lemme trigger one right now.
But wait, aren't you using it as a Magisk module? It wouldn't be of any significance to you. The "no more unnecessary wipes" change has been there since december 31.
Also, can you confirm regarding "not wiping": is it supposed to be not wiping when upgrading the ROM, upgrading MinMicroG or both?
Both. (upgrading just the ROM without flashing mmg manually shouldn't have been resetting permissions before anyway?)
Just in case -- upgrading minmicrog means flashing a new minmicrog zip from twrp, unless there is another way??
That or from the Magisk Manager.
OK. New naming scheme on test builds I guess?
I'm using MinMicroG-NoGoolag-UPDATELY-20210125200017.zip
For a test I have oneplus3 which is not an AB device, to illustrate the problem and avoid any potential conflict with A/B scenarios.
Magisk 21.4 and twrp 3.5.0_90 already installed. This particular device had MicroG 2.8 installed (i.e. latest release not nightly).
Scenario1: a ROM update test -- reboot to twrp, reflash with LOS nightly from 25012021 (i.e. lineage-17.1-20210125-nightly-oneplus3-signed.zip), magisk addon automatically triggers as expected. Wipe cache/dalvik -- reboot to system -- all works.
Scenario2: now that the rom has been updated, we reboot to twrp again. Only the new updately minmicrog zip is flashed, nothing else. Then reboot to system. And the very first message after authentication -- allow lawnchair to access photos, media, and files on your device? which tells us that app permissions had been reset.
@pwn0r insert Bernie meme
Do you see a
/data/.mmgfile in your recovery? Do you see anything aboutPREPPERin the recovery logs of the flash?
I think you might have missed one of my messages (5th above)
@pwn0r insert Bernie meme
Do you see a
/data/.mmgfile in your recovery? Do you see anything aboutPREPPERin the recovery logs of the flash?I think you might have missed one of my messages (5th above)
Not sure about recovery but on the system I can see /data/.mmg -- cannot open the file no permissions.
Where are the logs? Magisk log seems to be in /cache folder but does not contain anything interesting, as I think it is renewed every reboot and new module install?
Not sure about recovery but on the system I can see /data/.mmg -- cannot open the file no permissions.
Seems good.
Where are the logs?
The recovery logs, since you flashed it from the recovery. There's a button in the Advanced menu of TWRP to copy logs to recovery.log in either internal storage or sdcard.
You have to get the logs in the same session when you flash the zip.
aha, actually thought so -- reflashed UPDATELY in the recovery. PREPPER is present now and so all the permissions.
Package: NoGoolag
Version: UPDATELY
Release date: 25 Jan 2021
Using architecture: arm64
Using SDK level: 29
Sysroot is on /system_root
Mounting...
Using /data/adb/modules
Archive: /sdcard/MinMicroG-NoGoolag-UPDATELY-20210125200017.zip
inflating: util/funcs.txt
Doing MicroG preparations...
PREPPER: This is an update flash
I have a theory that permission fix only works past 2.8 version?? Because on both devices when updating from 2.8 (for 17 Jan and 25 jan test builds) it triggered permissions reset (and unselecting all location providers in the microg UI as well).
@pwn0r Hmm, that's certainly strange. It shouldn't have reset perms even updating from the older version, and that was the case for my own and Terminator's tests.
Maybe some other factor was involved? Data wasn't decrypted or something?
You can try flashing this last build a few more times under different condition if you wish, to make sure that it's all working properly now.
Otherwise I guess, Yay! Issue resolved?
data is permanently decrypted on both devices. if anything that should make it easier, or there is another factor?
(on A/B thats really a requirement as I had serious troubles early on when I first installed LOS on one7p. The fundamental problem is that not only system but firmware is specific per slot. Under certain circumstances if one slot had firmware too old, then reboot after update may soft-brick the device).
of course, lets call it resolved for now, I can check later when a new version is released.
P.S. for those looking how to properly disable encryption which is apparently enforced in LOS17.1 on A/B devices, you should refer to https://github.com/Zackptg5/Disable_Dm-Verity_ForceEncrypt
The dev also has ready-made zip files which need to be flashed last (before booting to system).
Gotcha. Just ping here if you wish the issue to be reopened.