Publish on F-Droid
Integral-Tech opened this issue · 95 comments
It would be better if ServerBox is submitted to F-Droid.
That's a good idea. Could you please help me submit this app to F-Droid, or should I do it myself?
I have submitted an issue to request for packaging (https://gitlab.com/fdroid/rfp/-/issues/2780)
Your main git submission email is used for filling AuthorEmail
Thank you for your efforts. :)
I have read the doc and it states that a Fastlane config should be created inside this repo.
I would like to get more people to contribute, so could you open a PR for that?
Thank you for your efforts. :) I have read the doc and it states that a Fastlane config should be created inside this repo. I would like to get more people to contribute, so could you open a PR for that?
Yes, I have opened a pull request
In the RFP issue, the bot detected non-free dependencies (seems to be GMS-related), which should be removed:
- com.google.android.c2dm.permission.RECEIVE
There are also some dependencies / services that could be used to track users, which should be minimized:
- ly.count.android.sdk.messaging.CountlyPushActivity
- ly.count.android.sdk.ReferrerReceiver
- https://firebase.google.com/support/privacy/init-options
- https://plus.google.com/
It's related to #379, i will solve it later.
trying to build 64ce363
these proprietary libs are still pulled in via https://github.com/Countly/countly-sdk-flutter-bridge/blob/24.4.1/android/build.gradle#L38 and https://github.com/Rexios80/watch_connectivity/blob/watch_connectivity-0.2.0/watch_connectivity/android/build.gradle#L52
trying to build 64ce363
these proprietary libs are still pulled in via https://github.com/Countly/countly-sdk-flutter-bridge/blob/24.4.1/android/build.gradle#L38 and https://github.com/Rexios80/watch_connectivity/blob/watch_connectivity-0.2.0/watch_connectivity/android/build.gradle#L52
Thanks for the tip.
rebuilt from 5059872
2024-06-11 15:30:19,612 INFO: Successfully built version 1.0.963 of tech.lolli.toolbox from 5059872c3f7343054b702de8b629d53b303a8aad
2024-06-11 15:30:29,778 DEBUG: Checking build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:29,835 INFO: Scanning APK with dexdump for known non-free classes.
2024-06-11 15:30:29,951 DEBUG: > /opt/android-sdk/build-tools/33.0.0/dexdump /tmp/tmp2z8b7fta/classes.dex
2024-06-11 15:30:35,878 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallSessionState'
2024-06-11 15:30:35,901 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallStateUpdatedListener'
2024-06-11 15:30:35,927 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnSuccessListener'
2024-06-11 15:30:35,947 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnFailureListener'
2024-06-11 15:30:35,952 DEBUG: Problem: found class 'com/google/android/play/core/splitcompat/SplitCompatApplication'
2024-06-11 15:30:36,105 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallManager'
2024-06-11 15:30:36,118 CRITICAL: Found 6 problems in build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:36,120 ERROR: Could not build app tech.lolli.toolbox: Found blocklisted packages in final apk!
2024-06-11 15:30:36,120 DEBUG: Error encountered, stopping by user request.
which looks like a flutter bug https://gitlab.com/fdroid/fdroiddata/-/issues/2949
rebuilt from 5059872
2024-06-11 15:30:19,612 INFO: Successfully built version 1.0.963 of tech.lolli.toolbox from 5059872c3f7343054b702de8b629d53b303a8aad 2024-06-11 15:30:29,778 DEBUG: Checking build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk 2024-06-11 15:30:29,835 INFO: Scanning APK with dexdump for known non-free classes. 2024-06-11 15:30:29,951 DEBUG: > /opt/android-sdk/build-tools/33.0.0/dexdump /tmp/tmp2z8b7fta/classes.dex 2024-06-11 15:30:35,878 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallSessionState' 2024-06-11 15:30:35,901 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallStateUpdatedListener' 2024-06-11 15:30:35,927 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnSuccessListener' 2024-06-11 15:30:35,947 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnFailureListener' 2024-06-11 15:30:35,952 DEBUG: Problem: found class 'com/google/android/play/core/splitcompat/SplitCompatApplication' 2024-06-11 15:30:36,105 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallManager' 2024-06-11 15:30:36,118 CRITICAL: Found 6 problems in build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk 2024-06-11 15:30:36,120 ERROR: Could not build app tech.lolli.toolbox: Found blocklisted packages in final apk! 2024-06-11 15:30:36,120 DEBUG: Error encountered, stopping by user request.which looks like a flutter bug https://gitlab.com/fdroid/fdroiddata/-/issues/2949
But there are some Flutter apps submitted successfully to F-Droid, such as FluffyChat. How did they solve this problem?
Not sure what's their magic sauce :(
FluffyChat emptied their proguard-rules.pro.
@licaon-kter I have cleaned proguard-rules.pro. Try to rebuild it :)
Builds fine now
@lollipopkit up next, setup Flutter as a submodule (and use it yourself in your CI and build recipes) or at least have the exact version written somewhere
tag and build the APK from the tagged commit so we can test for reproducibility (ref: https://f-droid.org/docs/Inclusion_How-To/#reproducible-builds and https://f-droid.org/2023/09/03/reproducible-builds-signing-keys-and-binary-repos.html)
@lollipopkit up next, setup Flutter as a submodule (and use it yourself in your CI and build recipes) or at least have the exact version written somewhere
Thanks, i have two question:
- After read the ref, I can’t figure out where(
exact version written somewhere) to specify the Flutter version. Is it inpubspec.yaml? If possible, use the latest stable release of Flutter. - If fdroid builds with my sign key, must I write the plain passwd to metadata?
If possible, use the latest stable release of Flutter.
We need to pin the version. The latest stable release is changing.
If fdroid builds with my sign key, must I write the plain passwd to metadata?
We don't need your key. Do not send your key to anyone. :)
For example, there is the revision of the flutter in https://github.com/lollipopkit/flutter_server_box/blob/main/.metadata. If you keep the file up-to-date we can use the revision specified there.
After read the ref, I can’t figure out where(exact version written somewhere) to specify the Flutter version. Is it in pubspec.yaml?
Where you want, eg. some use the CI, say put it here: https://github.com/lollipopkit/flutter_server_box/blob/main/.github/workflows/release.yml#L52 but not stable but 3.22.1
We need to pin the version. The latest stable release is changing.
yes, eg. yesterday was 3.22.1 you tagged (example) a version and built it
F-Droid builds today, and today... surprise 3.22.2 is stable. Maybe it's not an issue between .1 and .2 but there are issues between major versions
We don't need your key. Do not send your key to anyone.
exactly, do read those two links 😄
Thank you. I understand.
I still have questions about how to build with my sign key (if possible)? Or to release with fdroid signature (only in fdroid)?
We only use your signature if we can make the build reproducible. See https://f-droid.org/docs/Reproducible_Builds/.
You already host apks like: https://github.com/lollipopkit/flutter_server_box/releases/tag/v1.0.949
$ apksigner verify --print-certs ServerBox_949_amd64.apk
Signer #1 certificate DN: CN=JunyuanFeng, OU=ToastStudio, O=ToastStudio, L=ChengDu, ST=Sichuan, C=86
so.. you're already setup, build and sign as usual, F-Droid will use your verified reproducible APK 😉
Sorry... I thought the apk released in fdroid would be built in fdroid ci, so was wondering how to sign it.
@lollipopkit look at the graph https://f-droid.org/docs/Reproducible_Builds/ it's YES and YES and ends up with "publish @lollipopkit's APK" and not "sign something"
It's my problem...
Before you explained it, i thought build reproducible could only be fdroid ci
Now the flutter version is specified in the .github/workflows/release.yml
is there a reason why the Tag name and the versionName don't match? https://github.com/lollipopkit/flutter_server_box/blob/v1.0.967/pubspec.yaml#L4
The version in pubspec.yaml is generated by a script before pushing, but the version in the tag is generated by CI after pushing. So there is a discrepancy.
Is this important? Should I fix it?
Yes, it's important because we need a correct tag name to get your apk.
eg: https://github.com/lollipopkit/flutter_server_box/releases/download/v1.0.967/ServerBox_967_amd64.apk
we can use versionCode and versionName as templates for your APK url
say https://github.com/lollipopkit/flutter_server_box/releases/download/v%v/ServerBox_%c_amd64.apk
but... if those do not match your Tag and are +1 we can't
regarding versionCode of the per arch APKs, please move the arch number at the end so instead of the current 1967/2967/4967 it's better to have 9671/9672/9674
it used to be here https://github.com/flutter/flutter/blob/40e3489254b2d0ffedf840bd9233e5e6d06d09dd/packages/flutter_tools/gradle/flutter.gradle#L749 kinda so instead of abiVersionCode * 1000 + variant.versionCode have it variant.versionCode * 10 + abiVersionCode
not sure where this was moved in 3.22.x
regarding versionCode of the per arch APKs, please move the arch number at the end so instead of the current 1967/2967/4967 it's better to have 9671/9672/9674
it used to be here https://github.com/flutter/flutter/blob/40e3489254b2d0ffedf840bd9233e5e6d06d09dd/packages/flutter_tools/gradle/flutter.gradle#L749 kinda so instead of
abiVersionCode * 1000 + variant.versionCodehave itvariant.versionCode * 10 + abiVersionCodenot sure where this was moved in 3.22.x
@licaon-kter I have fixed this in #391
v1.0.971 solved these two.
Ok, builds, some notes:
- the APK names, since versionCode is different per arch, maybe name them based on that instead being
971, as they are97139711etc, or better yet just remove the971 - it's almost repro, here's the diff log: tech9713.log.7z.zip (remove
.ZIPfrom filename)
@linsui this diff looks familiar?
/LE: recipe updated here: https://gitlab.com/fdroid/rfp/-/issues/2780#note_1946717714
No. @lollipopkit Can you explain how you build your apk?
I built the apks as shown in Actions.
The reason why this diff occurs, may due to (i guess):
Zulu jdk17:
My custom build script:
I want to know what your script does. :) Can we also run
dart run fl_build -p android
to build the apk?
Yes, dart run runs a pkg(bin) named fl_build, it can be found at:
flutter_server_box/pubspec.yaml
Line 79 in 108d0a5
Does it do anything special? How can I build only 1 abi with it?
It updates metadata used by update checker and others.
It can't build 1 abi...
I think just use zulu to build, then the diff will decrease much.
Zulu is only used to build the java part but the libapp.so has difference. The BuildData.build is different. In your apk it's 971 and in our apk it's 970.
Ok, i will update it to support only 1 abi.
But it can't completely repro.
The buildAt vary every build time.
Why is https://github.com/lollipopkit/flutter_server_box/blob/v1.0.971/lib/data/res/build_data.dart not updated to 971? Does your script update it when building the apk?
Does your script update it when building the apk?
Yes
That's the problem... We need to build the apk from the same code. If it update the code everytime we can't get the same apk.
Can i delete buildAt but leave build?
We both use fl_build, so the build would be the same.
I thought it should work.
Now the build script will output apk at build/app/outputs/flutter-apk/app-release.apk
How should we build, e.g., an arm64 apk with fl_build?
dart run fl_build -p android -ooa
How can I tell it which abi should be built?
Defaults to arm64 (-ooa), need other archs?
Is the versionCode in app name makes your script harder to write?
Since you publish 3 abis on GitHub...
fl_build without -ooa behaves as --split-per-abi, with 3 abis...
So i guess that you only need arm64 one...
Is the versionCode in app name makes your script harder to write?
If this matters you, i can delete the versionCode from app name.
So i guess that you only need arm64 one...
we need all 3, I mean not us... but users... users of arm need arm, users of amd64 need that
fl_build without -ooa behaves as --split-per-abi, with 3 abis...
so all 3 are created? we can use that, brb
If this matters you, i can delete the versionCode from app name.
yes, please. You can put versionName there, that's ok
Output format is the same as Actions assets.
so all 3 are created? we can use that, brb
Then every apk takes longer to build. So can you add an argument so that we can build the specified abi?
ok
+ dart run fl_build -p android
GITHUB_ENV is not set. Skip writing env.
Running beforeBuild...
Unhandled exception:
PathNotFoundException: Cannot open file, path = 'ios/Runner.xcodeproj/project.pbxproj' (OS Error: No such file or directory, errno = 2)
#0 _checkForErrorResponse (dart:io/common.dart:55:9)
#1 _File.open.<anonymous closure> (dart:io/file_impl.dart:381:7)
<asynchronous suspension>
#2 _File.readAsBytes.<anonymous closure> (dart:io/file_impl.dart:562:24)
<asynchronous suspension>
#3 _File.readAsString (dart:io/file_impl.dart:621:18)
<asynchronous suspension>
#4 changeAppleVersion (package:fl_build/utils.dart:52:22)
<asynchronous suspension>
#5 main (file:///home/runner/work/flutter_server_box/flutter_server_box/.pub-cache/git/fl_build-9d5c990c66ce7d7448f4580ae832c93b3cf11314/bin/fl_build.dart:64:3)
<asynchronous suspension>
can you untangle the need for the ios folder when building for android?
ok
I decided to put the metadata update logic before the commit, otherwise it's too coupled...
So just use your scripts.
waiting on the tag to test 👍
Now you can compare the diff with v976
APK name is still wacky 😛
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
APK name is still wacky 😛
fmt like ServerBox_arm64.apk is ok?
I want to keep versionCode in apk name if possible...
you can't put the real versionCode in name?
Ignoring the fact that tag is 976 and in name you've put 977, the real number is 9671...2...3... right?
fact that tag is 976 and in name you've put 977
Oh, it's a bug
put the real versionCode
But the users will be confused on 9761, can I use 976 (Name_976_arm64.apk) instead of 9761?
see above #381 (comment) versionName or real versionCode.
977 uses versionName
There are 2 hard problems in computing
- naming things
- cache invalidation
- counting
ERROR: Could not build app tech.lolli.toolbox: Unexpected version/version code in output; APK: '1.0.978' / '9783', Expected: '1.0.977' / '9773'
@ 161f536
Interesting enough build log says:
2024-06-17 12:52:12,375 DEBUG: buildserver > + dart run fl_build -p android
2024-06-17 12:52:13,781 DEBUG: buildserver > GITHUB_ENV is not set. Skip writing env.
2024-06-17 12:52:13,781 DEBUG: buildserver > Running beforeBuild...
2024-06-17 12:52:18,502 DEBUG: buildserver >
2024-06-17 12:52:18,502 DEBUG: buildserver >
2024-06-17 12:52:18,503 DEBUG: buildserver > [apk]
2024-06-17 12:52:18,503 DEBUG: buildserver > flutter build apk --build-number=978 --build-name=1.0.978 --split-per-abi
I see 977 here https://github.com/lollipopkit/flutter_server_box/blob/v1.0.977/lib/data/res/build_data.dart#L5 though....
Use your build script instead of fl_build in order to decouple these logics.
@lollipopkit yup, I've read it backwards 😛
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
@lollipopkit yup, I've read it backwards 😛
Wasn't this a client for my own servers? Why does it connect to cdn.lolli.tech at start? ref: https://github.com/lollipopkit/flutter_server_box/blob/v1.0.977/lib/data/res/url.dart#L2
It's for auto update. Auto update should be opt-in.
It's update checker url. There is a switch in settings, you can turn it off.
It needs to be off by default, specially for reproducible builds.
Can i add a mark (such as //FOR_FDROID) at line end:
And you replace it to false in your script?
How would the APK then be reproducible if the source is different?
...Add an intro screen at first launch for this settings setup?
That would be good. :)
It needs to be off by default, specially for reproducible builds.
Is it because the apk published by F-Droid has different signature from apk downloaded from GitHub Release?
It has the same signaure. But some users don't understand where the update is from. If the auto update is on by default, the user may switch to upstream channel without noticing that. And upstream devs may add non-free deps by mistake and push the update to users directly before F-Droid can check that.
Add an intro screen at first launch for this settings setup?
...that explains that the update is from the developer
@Integral-Tech two ideas
- as said above, expectations, users wanted the app "built and verified by F-Droid"
- if on by default means we need to add NonFreeNet and Tracking anti-features to the app, ref: https://f-droid.org/docs/Anti-Features/ (but the point above has priority anyway)
Add an intro screen at first launch for this settings setup?
...that explains that the update is from the developer
@Integral-Tech two ideas
* as said above, expectations, users wanted the app _"built and **verified** by F-Droid"_ * if on by default means we need to add **NonFreeNet** and **Tracking** anti-features to the app, ref: https://f-droid.org/docs/Anti-Features/ (but the point above has priority anyway)
Okay, I see
Sorry for late.
Is this okay?
Looks good to me. :)
992 has been released
for the next version do correct the typo (in all locales) https://github.com/lollipopkit/flutter_server_box/blob/v1.0.992/lib/l10n/app_en.arb#L104 it's F-Droid 😉
Thanks for your help.
🥳