No longer recognizing untracked files
simplyandyg opened this issue · 50 comments
Describe the bug
MGit is no longer recognizing untracked files on Android. I know that there is a filesystem issue due to new permissions in Android 11+, but isn't there a "see all files" permission? I know that there are a few apps that have it, such as Solid Explorer.
I have even moved my repo and files to the Documents folder as I saw recommended in some of the Google Play Store review replies. That did not solve the issue either. I have multiple new files that I created with a different app, and I am not seeing them show up in MGit as untracked files like I would expect.
To Reproduce
Steps to reproduce the behavior:
- Create a file with an app such as Obsidian or Markor in a file within the repo
- Open MGit
- Check for changes - no changes found
Expected behavior
After creating new files they should be seen as untracked files in MGit
Screenshots
If applicable, add screenshots to help explain your problem.
Smartphone (please complete the following information):
- Device: Google Pixel 6 Pro
- OS: Android 12
- MGit App Version 1.6.2
I have been trying to work through this as well. I am able to track modifications on existing files, or adding files through MGit and then modifying them. However any new file is being ignored.
Yes, same. If create the file through MGit, it works.
And for some reason, I'm now noticing that if there was a file named "ABC" already in the directory, even if I rename the file to something else like "XYZ", MGit won't let me create a new "ABC" file. No error, just fails silently. Unlike When I try to create a file that I know is there, I do get the error that a file with that name already exists.
I suppose it could be a weird interaction with Obsidian, but I don't think that should cause any issues. It should just create the file
Also experienced this on April 7, 2022
Android 11
Oxygen OS 11.0.4.BE83BA
Seeing the same thing; creating a new file for a pre-existing tracked file doesn't seem to pick up external changes, either.
Also experiencing this on Android 11 with One UI 3.0.
Thank you everyone for reporting this.
Unfortunately there is nothing I can do at the moment as I have applied for the "All File Permission" on Google Play for MGit and it was not granted. I have appealed that decision and am awaiting a result for that at the moment.
@maks I understand. Thank you for the update though. It's really going to be unfortunate if they don't grant you the permission, as your app is the best git tool I've found for Android.
I'm happy to download an APK and install it outside of Google play. I'm already doing that with termux and a few other apps. Are you open to posting the built APK on GitHub somewhere? (Pretty please!)
I'm happy to download an APK and / or email someone at Google if it will help persuade the Play Store to grant the permission.
I was also wondering about an APK. I would do it that way as well. Google not giving you this permission has really messed with my workflow
Thank you everyone for your support.
I currently am waiting to hear back from Google Play in regards to the appeal. I'm hesitant to distribute APK's myself but I will look at that option if it ends up not being possible to continue in Google Play.
I'm hesitant to distribute APK's myself but I will look at that option if it ends up not being possible to continue in Google Play.
fdroid is an option, i wouldn't mind downloading directly from github either though
Edit: I didn't know the app was already on fdroid
MGit is already on F-Droid, but even installed from there, it doesn't have the 'all files' permission it needs. I assume it needs to be added to the manifest, and then I guess a Play Store edition created without it.
I ended up setting up Termux and following an online guide to get command line git setup. It's annoying because Termux has its own permissions issues. But as long as you are okay with setting up your local repo in one of the places Termux does have access to then it's fine. It recognizes new files. You just have to be okay with doing things on the command line on a phone, which is its own special level of masochistic sometimes.
Would it be possible to use the Storage Access Framework for setting the base directory for your repositories? That's supposed to give you access to all of the files and directories under that one...
https://developer.android.com/training/data-storage/shared/documents-files#grant-access-directory
No, because JGit can't deal with that.
Unless there's something to be able to access it using java.io.File.
@jfmcbrayer Unfortunately not. I've looked into using the SAF in the past but there are couple of showstoppers:
- JGit would need to be rewritten to use SAF APIs
- The SAF is far too low performance for the needs of Git, eg. being able to get high speed access to large numbers of files.
Thanks, that's terribly unfortunate.
You have access to Download (not tested, info from system log, peoples write it isn't usable) and Documents (tested) directories. Just place Github folder in Documents directory.
@daimoniuma that's interesting. I'll try that out
I did put my repositories in Documents. One would expect this to work, but in my experience, on Android 12 (LineageOS 19.1), it does not. You can clone a repository to there with no problems, but changes made by other apps will either not be seen, or shown as deleted files (rather than modified).
Google might decline all files access. Usage of Android/data/package.name/files is enough. A little problem is you cannot set this path in MGit as internal file manager cannot open full data directory.
P.S. Diff tool works in Android 11 with Documents/Github for me.
Actually, after updating to 1.6.3 (F-Droid), the permissions manager lets me set "Allow management of all files" ("media only" is the default), and repositories in Documents work correctly.
Didn't know that F-Droid version has extended permissions. With that you can store your repos anywhere, not only in Documents.
Actually, after updating to 1.6.3 (F-Droid), the permissions manager lets me set "Allow management of all files" ("media only" is the default), and repositories in Documents work correctly.
This worked for me also.
I hadn't noticed it was available as mgit already requested file access permission.
After manually setting it to "All files" I can add, diff and commit new changes.
Thanks for reporting that 1.6.3 is now available from F-Droid. That version has the "All Files Permission" added but yes it still requires users to go to the settings and allow that permission for MGit.
I'll add a dialog to prompt users to allow that in the next release.
Unfortunately Google Play has so far refused to allow MGit to use the permission, so a version with the permission is still not available on Google Play.
Is it possible for the community to help with the appeal (by collecting signatures or some kind of reassurance in a form issued by Google/the play store) ?
Idk if a petition would help but I would definitely sign one.
Personally I prefer getting from fDroid. Termux went this route and depreciated the play store version.
I feel everyone should slowly be relying more on fDroid where possible so I don't consider it a bad outcome either.
You can't collect a lot of voices even on change.org as it's a very special app for a really few peoples. You can improve settings for /Android/data path in GP version (you can't set this path in current version). You can detect install provider programmatically.
Thank you everyone for your support!
Unfortunately the appeal has been rejected and in both the initial rejection and appeal rejection, Google Play staff have not been helpful and have provided no specific reasons regarding the rejection.
I'm going to have a go at making some changes the apps UI to follow what other apps that use the "all files" permission use and resubmit the app then. If that's unsuccessful than I will be forced to remove the app from Google Play.
Actually, after updating to 1.6.3 (F-Droid), the permissions manager lets me set "Allow management of all files" ("media only" is the default), and repositories in Documents work correctly.
This worked. But it would be a good idea to document this anywhere. Maybe directly in the app, maybe on the readme page.
Maybe also this issue could be "pinned" and the solution could be added directly into the description of this current issue?
https://docs.github.com/en/issues/tracking-your-work-with-issues/pinning-an-issue-to-your-repository
If that's unsuccessful than I will be forced to remove the app from Google Play.
@maks, would you consider not removing it from Google Play? Despite the limitations, I use it day-in, day-out.
@klutometis I'm planning to make some UI changes first and resubmit the app on Google Play when I next get a chance. Depending on what happens after that I'll consider my options then if it is again rejected.
Although I haven't added untracked files that often so I haven't tried, is it a possible work around to use the "New File" command in MGit to create a blank file and then replace the content? Even this works, it is awkward to write non-text content... Is it possible for MGit to add a function like "Add File", and call an Android file manager, like com.google.android.documentsui , to list untracked file and add the select file to git?
@maks Maybe in your appeal you should point out to google that even though MGit is not aimed at modifying files from other application folders, other applications are to modify files in MGit folder. Thus MGit requires to be able to access files "from" other applications inside it own folder. (Ie also you could point them out that the editor is not part of MGit thus MGit requires to the ability to access files created by the editor application which is ... another application).
Other than them not understanding this point, I am out of guess why they would have refused MGit access to files created by other applications).
Also, there is the "access to all file types", you could point to them that in a git repository there are files that are not media files (especially with regards to #640 which I had me rename my ssh private key with a JPG extension to be able to import it in Android up to 11).
I am all for an FDroid repo workaround. But it is insane of Google to forbid VCS because if they do not give permission to MGit any VCS ought not to get any better luck.
I also pointed out the issue that MGit Google Play store version only allows "access to media files". There is the issue of not being able to access files that are not media files out of those MGit creates (the latter I suppose because I guess MGit is still allowed to create pdf, zip, etc., not only files with an extension of mp3, jpg, etc). But currently, you cannot import an ssh key file from MGit without renaming it with a JPG extension (then removing it via MGit UI after the import to a mgit subfolder succeed.
It is not an issue for me to have to create the MGit repo location under Documents. But being able to import only multimedia files is more of a problem. (I do not even think txt files are allowed to be imported).
Actually, after updating to 1.6.3 (F-Droid), the permissions manager lets me set "Allow management of all files" ("media only" is the default), and repositories in Documents work correctly.
This solution worked for me!
For folks not so familiar with Android, the "permissions manager" is in the Android system settings and not in the app settings. Also, at least on my OnePlus running Android 12, when you go "permissions manager" you are NOT provided with a list of apps to choose from, rather you are presented with a list of permissions categories. At that point choose "Storage" and then from there it sorts by level of permission, starting with apps that have full permissions. Scroll down to the apps that have limited permission, find MGit, click on it and then select "All Files."
Now MGit will recognize new files on its own.
I've made one last attempt at trying to make the app compliant with Google Play policies in regards to use of the "All Files access" permission and have resubmitted for review to Google Play.
At this point (as of #666) as far as I can see, MGit's use of the permission matches what VLC does and they have been approved on Google Play.
If after this latest round MGit is still rejected by Google Play staff I will sadly have to remove MGit from Google Play and only have it available via F-Droid and possibly other third party apps stores.
Sadly Google Play has once again rejected the latest version so that's the end of the line for MGit on Google Play.
I will shortly publish a new version on Google Play that will remove support for Android 11 onwards and display a message to that effect, referring users to instead install the app from F-Droid.
To help users who would need to upgrade vs uninstalling and re-installing from F-Droid (because F-Droid signs APK's with their own certificate and not mine) I will also begin publishing APK's from CI soon, though I would like to do that only once I have a update check mechanism in place (#669)
Besides of F-Droid and APK`s, is it possible to also make MGit available on Samsung's Galaxy Store? I control my phone by Google Family Link, which does not allow install apps by APK, but it allows me to install apps via preinstalled Galaxy Store...
@yuttai I'll try to look into this as I'm not sure what the requirements are to be able to publish on the Galaxy Store.
Mgit 1.7.0 from F-droid, on Android 13.0.0/Lineage 20 - "Clone failed, Operation not permitted" in both /storage/emulated/0/[Download|Documents].
It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue?
sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.
…
On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: @.>
Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.
How does one detect the file which makes Mgit to reject the whole downloaded repo? @maks
sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.
…
On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: _@**.**_>Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.
How does one detect the file which makes Mgit to reject the whole downloaded repo? @maks
It took me some time to figure it out.
Since the private repo I had the issues with is a collection of my notes, their names tend to contain a lot of special characters (in my case " and ?), which are invalid to use in filenames in the latest version of Android. I've just switched to a new phone, so I didn't know that.
The list of the invalid characters one can see here.
sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.
…
On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: _@**.**_>Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.
How does one detect the file which makes Mgit to reject the whole downloaded repo? @maksIt took me some time to figure it out.
Since the private repo I had the issues with is a collection of my notes, their names tend to contain a lot of special characters (in my case
"and?), which are invalid to use in filenames in the latest version ofAndroid. I've just switched to a new phone, so I didn't know that.The list of the invalid characters one can see here.
As the function name shows private static boolean isValidFatFilenameChar(char c) { these are invalid names to write to a FAT partition. These filesystems are used on Android on extension cards (SD).
If you write to the internal emmc these characters are OK.
That is why these are not MGit or even Android per see forbidden file characters. These characters are forbidden on Microsoft filesystems which are the only ones we can use on external media on Android (so that Windows users can read these external media).
Are there any updates or a work around for the situation regarding untracked files?
@CodeMazeSolver as told in the preceding comment there is no MGit untracked file bug.
Only if you install MGit from Google Store it will tag MGit as not having the rights to check for files other than media files. Access to other files requires Google to accept to grant this right and they refused (it looks insane to me as they granted this right to the application where it was not critical and here it is critical).
The only way out is to install from FDroid or any other store where MGit has been uploaded.
This is google decision to not allow MGit to access files on the storage. You should bug them if you want them to change that.
Ie the permission is not shipped in the application code. This is tagged on the application when uploaded to the Google Store. Thus why the same code uploaded to other stores works.
But this was explained in this thread already. Only a few last comments talk about an unrelated issue that was due to Android using Microsoft FAT format for external storage (SD cards) to let users read the cards under Windows and ... well FAT not supporting quite a lot of characters. Thus MGit is unable to check out the git repository on FAT FS when such a character is in a filename in the git files.