Binary Artifacts are un-auditable
Opened this issue · 6 comments
In the wake of the xz-utils news, I think people should be un-trusting of binary artifacts committed to version control. If Pack were to catch on, it would have a similar position in the software ecosystem as xz.
Pack depends on SCL. Their repos have issues disabled (why?), but I believe you are the same author, so I'm opening an issue here instead.
For each library, https://github.com/SCLOrganization/Libraries includes compiled binaries only (.o, .a files) - it's not clear how a user can reproduce these independently.
I don't mean to sound accusatory, but I believe that finding a way to avoid storing binaries in git would be a good move to improve the auditability and overall security posture of the project.
Hello, and thank you for the remarks. You are right about improving auditability.
SCL is a standard library for Pascal (Like Boost for C++) and is independent of Pack, and the developers do not wish for many public interactions.
Pack is an independent project and, if needed, uses its own code or binaries.
In fact, the mentioned binaries are not used in Pack except libkernel32.a that is gotten from msys64. Pack uses Its own builds Libraries and as an ease of use for Pascal developers, as not all have access to C build systems. But near each binary, there is a build script like Build.sh that anyone can use to build their own version. You can find version information and how they are linked at their corresponding unit, like UPackSQLite3LibraryStatic.pas.
Add any more questions you have about the building process, and it will be answered.
Could you document which specific source trees you used, for the binaries currently present in this repo, for sqlite and zstandard respectively?
In the sqlite tarballs I'm looking at, parse.y does not exist in the root directory, nor does lemon (it must be built first with make lemon), so I'm unable to follow build.sh as written.
I am guessing you are using the amalgamation download. That is a prepared source, and we need to configure it for Pack as it disables many features of SQLite to be safer and lighter.
SQLite: https://www.sqlite.org/2024/sqlite-src-3450100.zip (You can see in the .pas unit I shared that Pack currently uses 3.45.1. in future updates, if changed, you can get the latest from the SQLite download page: https://www.sqlite.org/download.html)
Zstandard: https://github.com/facebook/zstd/releases/download/v1.5.5/zstd-1.5.5.tar.gz (From https://github.com/facebook/zstd/releases/tag/v1.5.5 because Pack uses the latest 1.5.5 as shown here UPackZstandardLibraryStatic.pas)
Put each source in their corresponding directory near Build.sh (hint: .gitignore in each folder is there for this reason) and run Build.sh.
Similar process in Linux and Windows.
Thanks for the links, but I'm still running into the same issue with lemon and parse.y. I don't mean to be obtuse - I think I can guess how to work around it (make lemon; ./lemon src/parse.y perhaps) - but I'd like to make sure I'm following the "official" build process properly.
I'd like to be helpful too - if I set up Github Actions runners to automatically build release binaries (or perhaps just these library object files) from sources, would you accept that as a PR?
You found an issue where you needed to run the script twice. I updated the SQLite3 Build.sh. Thank you for the report.
I appreciate wanting to help. I will be happy to look at a PR. I may change it or do a new version based on it, but it will certainly be helpful.
Steps:
- Download the sources. SQLite3 (Currently 3.45.1), Zstandard (Currently 1.5.5). Hint for the future: Check UPackSQLite3LibraryStatic.pas and UPackZstandardLibraryStatic.pas for the current version and download sqlite-src-XXXXXXX from SQLite Download page and zstd-X.X.X from Zstandard releases page
- Extract and keep the source directories at their corresponding locations in Libraries and in Build directory near Build.sh. Names are sqlite and zstd for the source directories, so Build.sh can find them.
Linux:
- You can use any Linux, but for Ubuntu:
- sudo apt update
- sudo apt upgrade
- sudo apt install binutils
- sudo apt install build-essential
- Run Build.sh from Build folder.
Windows:
- Get the latest msys2 and then:
- pacman -S mingw-w64-ucrt-x86_64-gcc
- pacman -S base-devel
- pacman -S tcl (for SQLite)
- Using MSYS2 UCRT64 Run Build.sh from Build folder using
P.S. For these tests, I need to extract .zip and .tar.gz of the sources, and I am tired of waiting for them to extract.