opengapps/aosp_build

No source files specified with Andorid X86

dwtcheung opened this issue · 35 comments

I'm trying to include OpenGAPPS with Android X86. I read issue #217 and #229 and tried the following to resolve the issue but still running into the following error:

[570/645] including vendor/opengapps/build/modules/ActionsServices/Android.mk ... tools/test/connectivity/Android.mk: error: ActionsServices: No source files specified build/make/core/prebuilt_internal.mk:35: error: done. 17:37:32 ckati failed with: exit status 1

  1. Follow the instruction and modified the manifest and device.mk files
  2. Install git lfs
  3. Run repo sync
  4. Go into each of the folders inside opengapps/sources and run git lfs install and git lfs pull as sudo and got more files downloaded
  5. Try to compile and got the above error. Modified the find-apk.sh file to change lzip to refer to /usr/bin/lzip, still have the same problem

Not really sure what to do or how to diagnose the problem (Linux noob). The patch inside #217 doesn't seem to apply any more since the find-apk.sh file was updated and the ls -v command was removed.

Hello
i've gotten the same outcome as you did

Me too. I was trying to build the aosp emulator with gapps.

I ran into this issue a while earlier. If I'm guessing correctly, ActionServices is supposed to be APKs residing in sources/$ARCH/priv-app/com.google.android.as where $ARCH is one of arm, x86 etc. However it turns out that com.google.android.as only exists for arm64 and arm, but not x86.

The design is actually properly documented on https://github.com/opengapps/opengapps/wiki/Notes-for-Android-x86, but the actual implementation does not seem to take this in consideration.

I've isolated three find_apk.sh command leading up to this error. Here are the parameters to each command:

'vendor/opengapps/sources' '28' 'mdpi' 'com.google.android.as' 'all' 'false' 'prebuilts/sdk/tools/linux/bin/aapt'
'vendor/opengapps/sources' '28' 'mdpi' 'com.google.android.as' 'x86_64' 'false' 'prebuilts/sdk/tools/linux/bin/aapt'
'vendor/opengapps/sources' '28' 'mdpi' 'com.google.android.as' 'x86' 'false' 'prebuilts/sdk/tools/linux/bin/aapt'

The build system aparently did not bother checking arm / arm64 apks when ActionServices in question wasn't available for our build target architechture amd64.

What I would suggest at this point is probably run find_apk.sh one more time against arm even if the build target was x86_64 or x86.

It seems the same issue also exists when searching for shared objects. I'm attempting a patch on prebuilt_apk.mk and prebuilt_shared_library.mk.

Here's a dirty and quick patch that at least allowed me to build Android x86 Pie with PICO set of gapps.

From 0133f951b066185ff0c0ccf004c7dda50e606cfb Mon Sep 17 00:00:00 2001
From: Tianhao Chai <cth451@gmail.com>
Date: Wed, 11 Dec 2019 10:25:33 -0600
Subject: [PATCH] Fix prebuilt apk arch selection

When building against x86_64 and x86, the build system should select
APKs for arm when an APK is not available for x86, x86_64 or all.
---
 core/prebuilt_apk.mk | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/core/prebuilt_apk.mk b/core/prebuilt_apk.mk
index 2018d94..ee60b57 100644
--- a/core/prebuilt_apk.mk
+++ b/core/prebuilt_apk.mk
@@ -42,6 +42,10 @@ else
       ifdef LOCAL_SRC_FILES
         LOCAL_MODULE_TARGET_ARCH := $(TARGET_2ND_ARCH)
         LOCAL_PREBUILT_JNI_LIBS_$(TARGET_2ND_ARCH) := $(call find-libs-in-apk,$(TARGET_2ND_ARCH),$(LOCAL_SRC_FILES))
+      else
+        LOCAL_SRC_FILES := $(call find-apk-for-pkg,arm,$(LOCAL_PACKAGE_NAME))
+        LOCAL_MODULE_TARGET_ARCH := arm
+        LOCAL_PREBUILT_JNI_LIBS_arm := $(call find-libs-in-apk,$(TARGET_2ND_ARCH),$(LOCAL_SRC_FILES))
       endif
     endif
   endif
-- 
2.23.0

Hi all,
I build Android x86 pie with this patch but when use play store it does not work.

Supplement: I used userdebug mode.

Error message:

03-05 08:11:15.191 11936 11936 W DeviceDoctorHandler: Crash Hash: fc2387b3f0913e05cb34e34f85c6cc46a0d7876b
03-05 08:11:15.194 11936 11936 W DeviceDoctorHandler: Shushing crash.
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: FATAL EXCEPTION: main
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: Process: com.google.android.gms.persistent, PID: 11936
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: java.lang.RuntimeException: Unable to bind to service com.google.android.location.network.NetworkLocationService@31d16a8 with Intent { act=com.android.location.service.v3.NetworkLocationProvider cmp=com.google.android.gms/com.google.android.location.network.NetworkLocationService }: java.lang.SecurityException: Failed to find provider com.google.settings for user 0; expected to find a valid ContentProvider for this authority
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.app.ActivityThread.handleBindService(ActivityThread.java:3575)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.app.ActivityThread.access$1400(ActivityThread.java:199)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1671)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Handler.dispatchMessage(Handler.java:106)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Looper.loop(Looper.java:193)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.app.ActivityThread.main(ActivityThread.java:6669)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at java.lang.reflect.Method.invoke(Native Method)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:518)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: Caused by: java.lang.SecurityException: Failed to find provider com.google.settings for user 0; expected to find a valid ContentProvider for this authority
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Parcel.createException(Parcel.java:1950)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Parcel.readException(Parcel.java:1918)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Parcel.readException(Parcel.java:1868)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.content.IContentService$Stub$Proxy.registerContentObserver(IContentService.java:779)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.content.ContentResolver.registerContentObserver(ContentResolver.java:1972)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.content.ContentResolver.registerContentObserver(ContentResolver.java:1961)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at aelv.a(:com.google.android.gms@200414032@20.04.14 (100800-294335909):3)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at aely.a(:com.google.android.gms@200414032@20.04.14 (100800-294335909):94)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at aely.a(:com.google.android.gms@200414032@20.04.14 (100800-294335909):91)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at com.google.android.location.network.NetworkLocationChimeraService.onBind(:com.google.android.gms@200414032@20.04.14 (100800-294335909):5)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at qli.onBind(:com.google.android.gms@200414032@20.04.14 (100800-294335909):4)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.app.ActivityThread.handleBindService(ActivityThread.java:3559)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	... 8 more
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: Caused by: android.os.RemoteException: Remote stack trace:
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at com.android.server.content.ContentService.registerContentObserver(ContentService.java:340)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.content.IContentService$Stub.onTransact(IContentService.java:76)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at com.android.server.content.ContentService.onTransact(ContentService.java:262)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 	at android.os.Binder.execTransact(Binder.java:731)
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 
03-05 08:11:15.194 11936 11936 E DeviceDoctorHandler: 
03-05 08:11:15.201 11936 11936 I Process : Sending signal. PID: 11936 SIG: 9

How can i fix it?

Thanks for your help.

I guess the patch is still missing some part of the google framework. I'll do a build when I have more time.

This issue remains. Is there a walkaround?

system/sepolicy/Android.mk:79: warning: BOARD_SEPOLICY_VERS not specified, assuming current platform version
[968/1044] including vendor/opengapps/build/modules/ActionsServices/Android.mk ...
zipinfo: cannot find or open , .zip or .ZIP.

@zhendong2050 I can't remember how I resolved that problem. But make sure in your manifest file, you add ARM and ARM64 back in there. Install lzip (lower case L) and follow my steps in the initial post. You have to go into the all the sub folders in source and pull down the files. Very annoying since it's taking up more space on my HDD.

@dwtcheung since this is not a bug but only usage trouble and you have resolved it already with git-lfs on your own you should close this issue.

@zhendong2050 you have to do a git lfs pull in the opengapps directory to replace the placeholder files with the actual blobs.

Here an example how to do it:
https://github.com/stefanhh0/aosp-10/blob/e3f8e5dd640722d11c9690f1d18de7b56f29c759/build-4.14.sh#L184..L193

Following ones duplicate the usage problem: #251 and #253

Best regards,
Stefan

@dwtcheung @stefanhh0

Thanks for your reply.
I am trying to create an emulator with lunch mini_emulator_x86-userdebug.

Before make, I run the following script as @stefanhh0 suggested and downloaded APKs and install ed lzip. However I still got this error during compiling. Can you please give me further help? thanks a lot.

Error:
system/sepolicy/Android.mk:79: warning: BOARD_SEPOLICY_VERS not specified, assuming current platform version
[934/1010] including vendor/opengapps/build/modules/ActionsServices/Android.mk ...
zipinfo: cannot find or open , .zip or .ZIP.
tools/tradefederation/core/util-apps/WifiUtil/Android.mk: error: ActionsServices: No source files specified
build/make/core/prebuilt_internal.mk:35: error: done.
19:15:41 ckati failed with: exit status 1

Script:
for _path in
vendor/opengapps/sources/all
vendor/opengapps/sources/arm
vendor/opengapps/sources/arm64
do
pushd $_path
git lfs pull opengapps-gitlab &
popd
done

Try this:

Download OpenGapps

  1. cd into your vendor folder
  2. mkdir -p opengapps
  3. cd opengapps
  4. mkdir -p sources
  5. cd sources
  6. sudo apt install git-lfs
  7. git-lfs install
  8. git clone https://gitlab.opengapps.org/opengapps/all.git (make sure it's around 1-2GB)
  9. git clone https://gitlab.opengapps.org/opengapps/arm.git (make sure it's around 20GB)
  10. git clone https://gitlab.opengapps.org/opengapps/arm64.git (make sure it's around 20GB)
  11. cd ..
  12. git clone https://github.com/opengapps/aosp_build.git
  13. mv aosp_build build

If you need x86 and x86_64 then git clone that too

@Rybo713
Thanks for your solution. I got the following error when executing

git clone https://gitlab.opengapps.org/opengapps/arm.git

Did you have this error before, any suggestion? thanks a lot.

Error downloading object: app/com.google.android.apps.photos/21/nodpi/4846836.apk (caec68b): Smudge error: Error downloading app/com.google.android.apps.photos/21/nodpi/4846836.apk (caec68bb61eeb5626d22f48d9c112d9447c3b96b6ed53329373da7ea44852b59): batch response: Post https://gitlab.opengapps.org/opengapps/arm.git/info/lfs/objects/batch: dial tcp 78.46.46.59:443: connect: connection refused

Errors logged to /home/zhendong/AOSP/vendor/opengapps/sources/arm/.git/lfs/logs/20200407T224855.98017133.log
Use git lfs logs last to view the log.
error: external filter git-lfs smudge -- %f failed 2
error: external filter git-lfs smudge -- %f failed
fatal: app/com.google.android.apps.photos/21/nodpi/4846836.apk: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

@zhendong2050 I haven't had this error before. Try using git checkout -f HEAD

@Rybo713 same error again using git checkout -f HEAD.
I need to look into this error.

@zhendong2050 Try deleting everything related to opengapps and trying again. I just tried it and it worked with no issues.

@Rybo713
BTW, the path specified in step 1 should be exactly same and cannot be changed?

@zhendong2050 It should just be in the vendor folder

@Rybo713
Right there is a vendor folder under build_directory. I did all operations since step 2 in that folder. I am not sure if there is something wrong with git lfs.

@zhendong2050 You had it in the right place before: /home/zhendong/AOSP/vendor. If there is an opengapps folder, remove it and start from step 2 again. U can also try reinstalling git-lfs too.

@stefanhh0 Please read @cthbleachbit comments. I just re-tested it and I'm still having the same error message. Here's what I done.

  • Pull the source files on March 5th, 2020, so it should be 8cc99dd.
  • Go into each of the sources folder (all, arm and X86_64) and ran git lfs install and git lfs pull as sudo
  • Build against Android 8 (x86 platform), no errors
  • Copy the same OPENGAPPS source files to Android 9 source folder
  • Build against Android 9 without cthbleachbit's work around, immediately throws the same error as my first post
  • Build against Android 9 with cthbleachbit's work around, the same error was not encountered (I'm compiling the Android 9 X86 image as I type)

So this is not a git lfs usage error. If it's an usage error then I expect to encounter the same or similar missing source error when I build against Android 8. The fact that it's happening when I'm building against Android 9 makes me think cthbleachbit is correct. There's something wrong with the source files, either in the prebuild script or some where else that's causing this to happen.

TLDR; I'm not closing this issue yet. Please review cthbleachbit's work around and determine if it's the right fix. If it is, then update the documentation and the script in the source.

Hello @zhendong2050 ,

which version of git lfs are you using, please check with: git lfs --version
I am personally running: git-lfs/2.10.0 (GitHub; linux amd64; go 1.13.8) and with that version it works fine. Maybe you can identify any bug reports with the version you are running and that have been already fixed.

The error:

Error downloading object: app/com.google.android.apps.photos/21/nodpi/4846836.apk (caec68b): Smudge error: Error downloading app/com.google.android.apps.photos/21/nodpi/4846836.apk (caec68bb61eeb5626d22f48d9c112d9447c3b96b6ed53329373da7ea44852b59): batch response: Post https://gitlab.opengapps.org/opengapps/arm.git/info/lfs/objects/batch: dial tcp 78.46.46.59:443: connect: connection refused

Seems to me more a problem of the git lfs client or a firewall/internet connection problem rather than a configuration problem with your git lfs repos. Try connecting via telnet 78.46.46.59 443 it should show something like:

stefan@mars:~/android/source$ telnet 78.46.46.59 443
Trying 78.46.46.59...
Connected to 78.46.46.59.
Escape character is '^]'.
^CConnection closed by foreign host.

It could also be a temporary problem with the host, have you retried? Are at least some files downloaded to completion? That would then be an indication of a flaky internet connection.

Best regards,
Stefan

Hello @dwtcheung,

you are writing:

  • Go into each of the sources folder (all, arm and X86_64) and ran git lfs install and git lfs pull as sudo

I think you should have instead: all, x86 and x86_64 but not arm. ARM arch is not meant to be used for x86 or x86_64, right?

Please retry using following git repos:

While the patch outlined in #231 (comment) may work during build-time it wouldn't work during runtime on all x86 devices. Only on devices that have proper ARM emulation on the x86 hardware. Even if you have ARM emulation in place I am unsure if that is something that can be assumed as granted on every x86 device. Thus I believe the fix wouldn't be accepted by the maintainers of this project. However you are free to create a PR for that if you have good reasons for it. And anyway I am not an expert for x86 phones, maybe ARM can always be executed on x86 hardware?!

Best regards,
Stefan

Hi @stefanhh0 , thanks for the quick reply.

Initially, I only pulled all, x86, and x86_64 source files and that's when I encountered the problem. With cthbleachbit's work around, I wasn't sure if I need to pull arm as well but since it referred to it, I pulled it just in case. Android x86 does have an ARM emulator but I haven't tried building an ARM version of OPENGAPPS and use it with the ARM emulator. I'll schedule a test build this weekend.

BTW, what does PR stand for? Not a developer myself so can't say I understand all the shorthand references.

PRs are pull requests.

The link from @cthbleachbit is pretty helpful:
https://github.com/opengapps/opengapps/wiki/Notes-for-Android-x86

I haven't read it before close enough, in fact I was a little to quick saying that this issue should be closed since it is a git-lfs problem. Now I have realized that it is not. Maybe you could improve the headline of the issue and your first comment a little with the knowledge you now have. I am sorry for my quick shot earlier, I should have read it more carefully in the first place.

Hi @stefanhh0 ,
The version of my git-lfs is git-lfs/2.10.0 (GitHub; linux amd64; go 1.13.4)

The downloading error is encountered when I was following instructions suggested by @Rybo713 . I did some tweak (git config credential.helper manager) and the error was gone.

Unfortunately, I failed to compile and the error is the same as I reported earlier. My lunch option is lunch mini_emulator_x86-userdebug and Android version is Android 9.

One questions: how large are the downloaded folders x86 and x86_64?

@zhendong2050 I got 13GB for x86 and 7GB for x86_64. Also my git-lfs is version 2.8.0 / 1.12.10 if that makes a difference

@Rybo713 I checked my folders, which are the same as yours 13GB for x86 and 7GB for x86_64. Are you building Android 10 or 9? Is it related to Android version?

@zhendong2050 I'm building Android 10. Not sure if it makes a difference tho

Is there a tutorial for installing Gapps apks into an emulator without doing compilation?

I installed Gapps into an emulator following a tutorial but it did not work well. It kept reporting the "Google play service is stoping" error.

Thanks.

There's a solution to this! See this ticket:

#257

I followed the instructions as stated in the above ticket and I managed to compile it with Android X86 9R2 without problem. Furthermore, I don't have any GMS issues as I did before. Apps from Google Store installed and usable. While cthbleachbit script mod worked, it still required you download the arm source code while cwhuang solution doesn't. Closing ticket as it's now obsolete. I still don't know if cwhuang's changes were merged to main line.