Reproducible Builds
Closed this issue · 16 comments
I've checked your app if its build is reproducible (see: Reproducible bulds, special client support and more in our repo), but I wasn't even able to build the app at all, getting the following error:
> Could not resolve all files for configuration ':app:releaseCompileClasspath'.
> Could not resolve com.nerdoftheherd:android-dropbear:2024.85.
Required by:
project :app
> Could not resolve com.nerdoftheherd:android-dropbear:2024.85.
> Could not get resource 'https://maven.pkg.github.com/ribbons/android-rsync/com/nerdoftheherd/android-dropbear/2024.85/android-dropbear-2024.85.pom'.
> Username must not be null!
> Could not resolve com.nerdoftheherd:android-rsync:3.3.0.
Required by:
project :app
> Could not resolve com.nerdoftheherd:android-rsync:3.3.0.
> Could not get resource 'https://maven.pkg.github.com/ribbons/android-rsync/com/nerdoftheherd/android-rsync/3.3.0/android-rsync-3.3.0.pom'.
> Username must not be null!
We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that. Though of course first this general build issue needed to be solved…
Looking forward to your reply!
This is due to the lovely behaviour of GitHub only allowing access to public package registry packages for authenticated users 🤦 (see ribbons/android-rsync#42 for some earlier context) - the Dropbear and rsync binaries are published (by me) to GitHub package registry and fetched as dependencies of this project.
As a workaround, you can set the Gradle gpr.user and gpr.key properties or the environment variables PKG_USERNAME and PKG_TOKEN (depending on which is most convenient for rbtlog), these are picked up by
build.gradle.kts.
I have the properties set in gradle.properties on my local machine and use the environment variables for the GitHub Actions CI and release process.
That sadly won't work with the rebuilder. Environment variables are not propagated into the container, and I won't put credentials in plain text to the recipe, which is public. Would there be no alternative – like, having them in a git submodule?
Okay, that's unfortunate. Re: a submodule - sadly not as the other repos set up the necessary toolchain, then build and package the binaries (which is what are needed in this app).
I understand that publishing to e.g. Maven Central would cause more work for you. But being unable to build the app without a GitHub account is a blocker for Reproducible Builds. And also somewhat in conflict with the FOSS principle that anyone should be able to build their own, without restrictions.
That said, from a quick look at the build scripts it seems the two dependencies aren't that hard to build (though the artefacts may not be reproducible -- e.g. if there is an embedded build ID -- without some changes). The main issue seems to be that the JAR generation code is in the github workflow. If you could factor that out into a separate script that can be used to build and publish not just to github as you do now but also alternatively to a local maven repository, we could use that during our build.
The issue then is how to match the version of those dependencies' repos against the version needed by the app, which is where having them as submodules as Izzy suggested would help.
After mulling it over (and discovering that GitHub no-longer seem to have any plans to enable anonymous access for their maven package registry, contrary to their earlier statements), I think I'll go down the route of attempting to publish those builds to Maven Central. That would have the benefit of also helping out folks with use cases like ribbons/android-rsync#42.
Will take me a while to get to this though as I'm rather mired in some other projects at the moment.
Thanks! And yeah, no rush, there are no "deadlines". We're looking forward to your ping whenever you found the time to tackle this. We might show up with a friendly heads-up every now and then, though, if you don't mind – not too frequent, or fourse 😃
Just a friendly heads-up: any news on this (or an ETA when we can continue with RB), @ribbons?
Just a friendly heads-up: any news on this (or an ETA when we can continue with RB), @ribbons?
That's a really odd coincidence... I was starting to fumble my way towards figuring out an actions job to publish the dropbear binaries to Maven Central yesterday evening! So yes, finally in progress 🙂.
Great news! More of such coincidences please wherever I send heads-ups 🤣
Okay, that should be a step closer to reproducible builds now, both of those dependencies are now published to Maven Central - took a bit of head-scratching along the way but feels good to have done it 😄.
Is it better if I go ahead and release a new version or would there be any utility in your re-running against the latest commit on the master branch (if that's possible)? Unfortunately one of the dependencies has been updated since the current release and I've only published the latest release of each to Maven Central.
You can attach an APK here (just rename it to *.zip so Github allows attaching it; if it's not packed into another ZIP I can run it directly from here) and name the commit it was built from, then I can run a check (including finding out if it's RB). I could of course also just try to see if I can build it at all, just specifying the latest commit, if you prefer that.
Either sounds good to me but here's an (unsigned) release build from the latest commit on master if that's any good 😄
Not sure, it's unsigned – so the rebuilder has no cert to copy over to its own build 😆
But well, let's see – rbtlog can also be used to simply build an app. Then I can check manually if the sha256sum matches … and: it does! So the app is RB at commit aca8621 – congrats! If you want to go ahead and make it a release, I can establish it in my builder 😃 Just let me know when it's there.
(if you think it's not a 0.14, a 0.13.1 would do as well 😉)
Hooray! Thanks for checking and that's great news 🥳.
I've just released 0.14 (decided it merited it as there was a minSdk bump included), so now with a bit of luck it should be fully reproducible 🤞.
Not RB but no diff… hm… ah, OK, then let's … --pad-like-apksigner 😜 Because:
file='9283544c8d86b6fa0d9abc9202d0e8944dd8ed349c8819ac2d7b1a47e55f86b8-com.nerdoftheherd.tasker.rsync-0.14-upstream.apk'
zipaligned (4-byte alignment) : yes
files with apksigner padding : 231
apksigner alignments from extra fields : 4
most likely uncompressed .so page alignment : none
(wonderful tools my sensei provided 😃) OK, another run… 🤞 Yeah, yeah, yeah – that was fun! All good things in live are… 3rd and final run, with debug stuff removed from the recipe – this time writing the logs… writing the index… add, commit, push, sync … TOOOOT!
Eh, and the first boost already before I could make the screenshot – popular app I'd say! Wish there was a FOSS pendant to Tasker supported by TaskerSync…
Thanks a lot, and congrats to the achievement!
Hooray and thanks for your help with this one! Keep up the great work 😄