LFS broke build
Closed this issue · 21 comments
While LFS is amazing it totally breaks building Gapps directly into ROM.
Android uses zipinfo to check the packages and well when using LFS it fails the check and causes build to fail.
FAILED: out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk
/bin/bash -c "(rm -f out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk ) && (cp "vendor/opengapps/sources/arm64/app/com.google.android.gms.policy_sidecar_o/23/nodpi/20440.apk" "out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk" ) && (if (zipinfo out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk 'lib/.so' 2>/dev/null | grep -v ' stor ' >/dev/null) ; then out/soong/host/linux-x86/bin/zip2zip -i out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk -o out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.tmp -0 'lib/**/.so' && mv -f out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.tmp out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk ; fi ) && (if ! out/host/linux-x86/bin/zipalign -c 4 out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk >/dev/null ; then mv out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.unaligned; out/host/linux-x86/bin/zipalign -f -p 4 out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.unaligned out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.aligned; mv out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.aligned out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk; fi )"
Unable to open 'out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk' for verification
Unable to open 'out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.unaligned' as zip archive
mv: bad 'out/target/product/mermaid/obj/APPS/AndroidPlatformServices_intermediates/package.apk.aligned': No such file or directory
Here is the error
This was with full wipe of all openGapps git files from sources and even from .repo folder. Repo sync still uses git clone even tho it requests git-lfs to be installed. And even when cloned manually with git-lfs clone it still makes the apks unusable.
Same here, the problem seems to be git-lfs. Yes, I deleted my source tree and did a fresh checkout into an empty directory. All apk files look like this:
$ ls -la vendor/opengapps/sources/all/priv-app/com.google.android.gsf/28/nodpi/
total 12
drwxr-xr-x 2 android android 4096 Nov 7 06:35 .
drwxr-xr-x 3 android android 4096 Nov 7 06:35 ..
-rwxr-xr-x 1 android android 132 Nov 7 06:35 28.apk
$ cat vendor/opengapps/sources/all/priv-app/com.google.android.gsf/28/nodpi/28.apk
version https://git-lfs.github.com/spec/v1
oid sha256:ca12b2320860e5dcda427b47c8edaeae8565b44a716e9299777f44a18077e2c8
size 4353352
EDIT:
Yes, I also ran "git lfs install".
Running "git lfs fetch --all" inside "vendor/opengapps/sources/arm" seems to download the real apk files however:
$ vendor/opengapps/sources/arm$ git lfs fetch --all
fetch: 238 object(s) found, done
fetch: Fetching all references...
Downloading LFS objects: 42% (100/238), 3.9 GB | 11 MB/s
It seems using "git lfs", but "git lfs pull", is expected after repo sync:
https://gerrit-review.googlesource.com/c/git-repo/+/83130
For any projects that have LFS objects, `git lfs pull` must be executed.
git lfs pull and even git lfs fetch --all downloads only lfs files still. IDK whats wrong anymore. I dont think it was such a good idea to switch to LFS.
Acually fetch does download apk files but to .git/lfs folder and the apks in real folders are not update accordingly.
@nezorflame Calling you since you seem to be working with openGapps so i think it is best to call you and not speculate.
@Haxk20 I've stumbled upon a similar issue with repo tool and LFS: git-lfs/git-lfs#1422
Try doing repo forall -c git lfs pull after the initial repo sync. This worked for me at least with a test project: https://github.com/nezorflame/lfs-test without any extra work:
~ ls -la vendor/opengapps/sources/all/priv-app/com.google.android.gsf/28/nodpi/
-rwxr-xr-x 1 nezorflame nezorflame 4353352 7 Nov 14:34 28.apk
Original mentioned fix also mentioned doing git lfs install --local --skip-smudge, but AFAIK that should've been fixed already. If the previous step doesn't work, I recommend to check that linked issue.
Well you basically just did git lfs pull on all repos which is totally fine but only gapps repos will download the files but i wonder if it will fix it as when i did git lfs pull on only sources/all it downloaded the files but only to .git/lfs folder where they stayed. But will tell you once its downloaded.
Well, as you can see from my answer, it did place the actual binary to the 28.apk after the pull. In any case, please let me know if the issue persists.
Will let you know tomorrow. In the process of switching to optic on home network and PFsense is not configured correctly still so it has drop outs. So will do it tomorrow.
I tested it and downloaded the objects but the actual APKs are not placed in the correct place after pulling and build still fails even after your command. @nezorflame
I tested it and downloaded the objects but the actual APKs are not placed in the correct place after pulling
I can not confirm this, after
for i in vendor/opengapps/sources/all vendor/opengapps/sources/arm vendor/opengapps/sources/arm64; do cd $HOME/android/los16.0/$i; git lfs pull; done
all apk files are present, yesterday I built a working lineageos-based ROM.
That is very very interesting. IDK whats wrong then
I will try with the test repo provided by @nezorflame and if that doesnt work then well IDK. @nezorflame Do you think it would be possible to have second repo for apk that are not LFS ? since if this doesnt work i truly dont know WTF is wrong.
I found out what was the issue. All that had to be done was git lfs install because i probably didnt install it. Well now we know. indeed all files are placed in the correct place and ready for building.
@Haxk20 glad you've figured it out!
We'll update the README a bit later with more details on how to properly set up LFS.
@Haxk20 glad you've figured it out!
We'll update the README a bit later with more details on how to properly set up LFS.
Yeah. Not really easy to set up TBH
I can't compile, I'm trying to do what you mentioned above, but I have this error
error: failed to fetch some objects from ''
batch request: missing protocol: "", 0 B | 0 B/s
Had the same error here.
A simple:
cd /vendor/opengapps/sources
Then a:
for d in ./*/ ; do (cd "$d" && git lfs pull); done
Fixed my problem. Glad I found this page. Thanks all