Automate release process

Question

Automate release process

stanhu opened this issue a year ago · 22 comments

Once #76 is merged, and the v2.0.x branch is released, we should probably automate the build and publish process, such as: https://github.com/y-crdt/yrb/blob/main/.github/workflows/gem.yml

Answer 1 · 2023-07-26T18:02:22.000Z

@mudge How would you like to proceed with a 2.x release? Would you like to see the automation set up first, or would you be okay with manually generating the packages and pushing them?

@flavorjones Just curious if you had any thoughts on automation. Does Nokogiri manually push based on https://github.com/sparklemotion/nokogiri/blob/main/CONTRIBUTING.md#making-a-release?

Answer 2 · 2023-07-26T18:23:01.000Z

@stanhu I release manually. It's a pretty low lift, and using RCD containers is repeatable, so I haven't felt a strong need to automate releases.

Answer 3 · 2023-07-26T18:46:22.000Z

I’m happy to push releases manually too. What does the process look like at this point?

Answer 4 · 2023-07-27T06:01:13.000Z

@mudge Normally I'd say:

Bump the version in version.rb.
Update CHANGELOG.md
Tag the version.
Build all the gems:

bundle exec rake clobber
bundle exec rake compile
bundle exec rake spec
bundle exec rake gem
bundle exec rake gem:native

Then push the tag and every .gem in pkg/.

However, I'm seeing some linker errors if I don't do a rake clobber before the x64-mingw32 build:

linking shared-object re2.so                                                                                                                                                                                                                                                                                                                                                
/usr/bin/x86_64-w64-mingw32-ld: /home/stanhu/github/re2/ports/x86_64-w64-mingw32/abseil/20230125.3/lib/libabsl_time_zone.a(time_zone_libc.cc.obj):time_zone_libc.cc:(.text+0x97): undefined reference to `__imp___timezone'
/usr/bin/x86_64-w64-mingw32-ld: /home/stanhu/github/re2/ports/x86_64-w64-mingw32/abseil/20230125.3/lib/libabsl_time_zone.a(time_zone_libc.cc.obj):time_zone_libc.cc:(.text+0xa8): undefined reference to `__imp___dstbias'
/usr/bin/x86_64-w64-mingw32-ld: /home/stanhu/github/re2/ports/x86_64-w64-mingw32/abseil/20230125.3/lib/libabsl_time_zone.a(time_zone_libc.cc.obj):time_zone_libc.cc:(.text+0xde): undefined reference to `__imp___tzname'
collect2: error: ld returned 1 exit status
make: *** [Makefile:262: re2.so] Error 1

I'm not sure why that's happening, since the libraries were just compiled in the ports/x86_64-w64-mingw32 directory.

Sometimes when I run bundle exec rake gem:x64-mingw32 locally I get this odd failure:

rake aborted!
Don't know how to build task 'native:x64-mingw32' (See the list of available tasks with `rake --tasks`)
/home/stanhu/github/re2/Rakefile:57:in `gem_builder'
/home/stanhu/github/re2/Rakefile:86:in `block (4 levels) in <top (required)>'
/usr/local/rvm/gems/ruby-3.1.3/bin/bundle:25:in `load'
/usr/local/rvm/gems/ruby-3.1.3/bin/bundle:25:in `<main>'
/usr/local/rvm/gems/ruby-3.1.3/bin/ruby_executable_hooks:22:in `eval'
/usr/local/rvm/gems/ruby-3.1.3/bin/ruby_executable_hooks:22:in `<main>'
Tasks: TOP => gem:x64-mingw32:builder
(See full trace by running task with --trace)
rake aborted!
Command failed with status (1): [docker run -v /home/stanhu/github/re2:/home/stanhu/github/re2 -e UID\=1001 -e GID\=1002 -e USER\=stanhu -e GROUP\=stanhu -e GEM_PRIVATE_KEY_PASSPHRASE -e ftp_proxy -e http_proxy -e https_proxy -e RCD_HOST_RUBY_PLATFORM\=x86_64-linux -e RCD_HOST_RUBY_VERSION\=3.1.4 -e RCD_IMAGE\=ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x64-mingw32 -w /home/stanhu/github/re2 --rm -i -t ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x64-mingw32 runas sigfw bash -c gem\ install\ bundler\ --no-document\ \&\&'
'bundle\ \&\&'
'bundle\ exec\ rake\ gem:x64-mingw32:builder\ CMAKE\=cmake'
']
/home/stanhu/github/re2/Rakefile:76:in `block (3 levels) in <top (required)>'
/home/stanhu/.asdf/installs/ruby/3.1.4/bin/bundle:25:in `load'
/home/stanhu/.asdf/installs/ruby/3.1.4/bin/bundle:25:in `<main>'
Tasks: TOP => gem:x64-mingw32
(See full trace by running task with --trace)

It seems to build fine when I do a bundle exec rake clobber, so there must be some odd dependency on a existing directory or filename. @flavorjones Have you seen this issue before?

Note it works fine in CI: https://github.com/mudge/re2/actions/runs/5634578432/job/15264669351?pr=76

Answer 5 · 2023-07-27T06:11:04.000Z

Here's the script I use to build nokogiri:

https://github.com/sparklemotion/nokogiri/blob/main/scripts/build-gems

I clobber between each of the platform types which probably avoids the behavior you're seeing. (I'm not sure what the root cause is.)

Answer 6 · 2023-07-27T07:43:04.000Z

Thanks, I did see that script. If I try to compile the Windows gems in a sequence, I get that unknown native:x64-mingw32 task:

bundle exec rake gem:x64-mingw-ucrt
bundle exec rake gem:x64-mingw32

One issue is that the Docker image 1.3.0-mri-x64-mingw32 only appears to support Ruby 2.4 to 3.0:

$ docker run -it ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x64-mingw32 bash
root@035b400d58f2:/# cat ~/.rake-compiler/config.yml
---
rbconfig-x64-mingw32-2.4.0: "/usr/local/rake-compiler/ruby/x86_64-w64-mingw32/ruby-2.4.0/lib/ruby/2.4.0/x64-mingw32/rbconfig.rb"
rbconfig-x64-mingw32-2.5.0: "/usr/local/rake-compiler/ruby/x86_64-w64-mingw32/ruby-2.5.0/lib/ruby/2.5.0/x64-mingw32/rbconfig.rb"
rbconfig-x64-mingw32-2.6.0: "/usr/local/rake-compiler/ruby/x86_64-w64-mingw32/ruby-2.6.0/lib/ruby/2.6.0/x64-mingw32/rbconfig.rb"
rbconfig-x64-mingw32-2.7.0: "/usr/local/rake-compiler/ruby/x86_64-w64-mingw32/ruby-2.7.0/lib/ruby/2.7.0/x64-mingw32/rbconfig.rb"
rbconfig-x64-mingw32-3.0.0: "/usr/local/rake-compiler/ruby/x86_64-w64-mingw32/ruby-3.0.0/lib/ruby/3.0.0/x64-mingw32/rbconfig.rb"
root@035b400d58f2:/# ls -al ~/.rake-compiler/ruby/x86_64-w64-mingw32/
total 28
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 .
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ..
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ruby-2.4.0
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ruby-2.5.0
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ruby-2.6.0
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ruby-2.7.0
drwxr-xr-x 1 rvm rvm 4096 Jan 12  2023 ruby-3.0.0

The latest snapshots in https://github.com/rake-compiler/rake-compiler-dock/pkgs/container/rake-compiler-dock-image have the same issue.

I think that's why we see these warnings (https://github.com/rake-compiler/rake-compiler/blob/100d4241f27fbc9568846a2c1d7a63febeef4522/lib/rake/extensiontask.rb#L398) in https://github.com/mudge/re2/actions/runs/5634578432/job/15264669351?pr=76:

 + bundle exec rake set-version-to-timestamp
no configuration section for specified version of Ruby (rbconfig-aarch64-linux-3.2.0)
no configuration section for specified version of Ruby (rbconfig-aarch64-linux-3.1.0)
no configuration section for specified version of Ruby (rbconfig-aarch64-linux-3.0.0)
no configuration section for specified version of Ruby (rbconfig-aarch64-linux-2.7.0)
no configuration section for specified version of Ruby (rbconfig-arm-linux-3.2.0)
no configuration section for specified version of Ruby (rbconfig-arm-linux-3.1.0)
no configuration section for specified version of Ruby (rbconfig-arm-linux-3.0.0)
no configuration section for specified version of Ruby (rbconfig-arm-linux-2.7.0)
no configuration section for specified version of Ruby (rbconfig-arm64-darwin-3.2.0)
no configuration section for specified version of Ruby (rbconfig-arm64-darwin-3.1.0)
no configuration section for specified version of Ruby (rbconfig-arm64-darwin-3.0.0)
no configuration section for specified version of Ruby (rbconfig-arm64-darwin-2.7.0)
no configuration section for specified version of Ruby (rbconfig-x64-mingw-ucrt-3.2.0)
<snip>

When I provided a "fake" entry for Ruby 3.2, I did notice the native task was created. Though I'm still not sure why a rake clobber seems to make it work.

Now, I'm puzzled how the .gem files have all the compiled extensions for all the supported Ruby versions. It seems that test-gem-install didn't actually have the right shell here: https://github.com/mudge/re2/actions/runs/5634578432/job/15264771818

Answer 7 · 2023-07-27T20:18:48.000Z

Ok, I see there are two issues here:

mini_portile installs the abseil C++ library for both x64-mingw-ucrt and x64-mingw32 in the same directory based on the host value (x86_64-w64-mingw32) in https://github.com/flavorjones/mini_portile/blob/ee637e0a76e5ae7144949d868948b45d8d858051/lib/mini_portile2/mini_portile.rb#L285-L291. As a result, if x64-mingw-ucrt is built first, then the abseil libraries will link against the Universal C Runtime (UCRT). The timezone, dstbias, and tzname symbols come from the Microsoft Runtime library (https://learn.microsoft.com/en-us/cpp/c-runtime-library/daylight-dstbias-timezone-and-tzname?view=msvc-170). When rake gem:x64-mingw32 is attempted after rake gem:x64-mingw-ucrt, then it will fail to link those symbols because the abseil library in ports/x86_64-w64-mingw32 directory expects UCRT to be used.

@flavorjones I'm guessing you don't run into that with Nokogiri because libxml2 and the other libraries don't link against anything in UCRT.

To avoid this collision, we might want mini_portile to support another subpath, such as RbConfig::CONFIG['sitearch']:

x64-mingw-ucrt ➡️ x64-ucrt
x64-mingw32 ➡️ x64-msvcrt

rake gem:x64-mingw32 fails to start due to Don't know how to build task 'native:x64-mingw32'. As explained in #80:

x64-mingw-ucrt only supports Ruby 3.1 and 3.2.
x64-mingw32 only supports Ruby 2.4 - 3.0.

When ENV['RUBY_CC_VERSION'] starts with a Ruby version that isn't supported by the image, the rake-compiler bails since it can't find the configuration in https://github.com/rake-compiler/rake-compiler/blob/100d4241f27fbc9568846a2c1d7a63febeef4522/lib/rake/extensiontask.rb#L398 and doesn't attempt to define the compiler tasks.

In any case, CI works fine since each job is built independently.

To handle releases, we could:

Write a build-gems script that will run rake clobber between each build (at least between Windows builds) to ensure there are no issues. The Nokogiri version retests these gems, which is probably a good idea, but it seems like CI already does this work, so why not take advantage of this (item 3)?
Drop x64-mingw32 support. Will anyone actually use this given it only supports up to Ruby 3.0?
Automatically publish from CI (or gather up artifacts) since those gems have been built in a clean environment and tested.

Answer 8 · 2023-07-27T20:42:42.000Z

Drop x64-mingw32 support. Will anyone actually use this given it only supports up to Ruby 3.0?

Just some data points from Nokogiri:

https://rubygems.org/gems/nokogiri/versions/1.15.3-x86_64-linux: 2,557,396 downloads
https://rubygems.org/gems/nokogiri/versions/1.15.2-x64-mingw-ucrt: 20,120 downloads
https://rubygems.org/gems/nokogiri/versions/1.15.2-x64-mingw32: 8,779 downloads

Answer 9 · 2023-07-27T21:36:55.000Z

Ok, #81 seems to fix both issues.

Answer 10 · 2023-07-27T22:11:46.000Z

@stanhu I'm on vacation and can't dig in on the mini portile paths, but your diagnosis sounds about right (and I've seen something similar in grpc cross builds). I can look when I'm back in a week or so.

Answer 11 · 2023-07-28T04:53:45.000Z

@flavorjones Thanks! I'm pretty sure this has solved the problem. I can build all native gems with no need to rake clobber between builds with #81.

@mudge With #81 and #82 (UPDATE: #83 needed too), the release process would be:

Bump the version in version.rb.
Update CHANGELOG.md
Tag the version.
Build all the gems via scripts/build-gems.
for g in gems/*.gem ; do gem push $g ; done

The script doesn't test the gem contents as Nokogiri's scripts do, but we can consider that later. I still wonder if it's easier to push the gems built from CI.

Answer 12 · 2023-08-10T07:38:47.000Z

@mudge Sorry to bother you again. Would you be able to cut a 2.x release?

Answer 13 · 2023-08-11T09:51:55.000Z

Hi @stanhu,

Thanks for your continued patience. As you may have noticed, I'm struggling to make time to properly review and test this at the moment.

However! I did just try running a build on the current v2.0.x branch on my M1 Max laptop and noticed the following warnings and errors during the run:

WARNING: The requested image's platform (linux/386) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
linux32: Failed to set personality to linux32: Invalid argument

I suspect the first is harmless and just means that Docker is running the images under virtualisation (see rake-compiler/rake-compiler-dock#78) but do you recognise the second?

Answer 14 · 2023-08-11T10:11:51.000Z

I'm wondering about how best to handle version 1 and 2 of the gem.

Is it high time I gave up on older Ruby versions, matching @flavorjones' policy of the last four versions (regardless of EOL status) and only offer 2.0 with its minimum Ruby version of 2.7 (even though the gem will work with older Rubies, it's just that the packaging of it does not)? If so, we could merge this into main and update the docs accordingly.

Or do I still want to maintain the version 1 branch, potentially pushing new versions if re2 requires it? In which case, there's some branching hygiene and documentation to update across main, v2.0.x (which should probably become v2) and a new v1 branch.

Sadly, I can't easily see what versions of Ruby current users are on but it is clear GitLab is the biggest dependent and it requires Ruby 3.0 so perhaps it is time to put version 1 to bed. Worst case, if there is something I need to backport, I can always dig it out of the git history later.

Answer 15 · 2023-08-11T11:05:02.000Z

My build finished with the following error:

rake aborted!
Command failed with status (1): [docker run -v /Users/mudge/Projects/re2:/Users/mudge/Projects/re2 -e UID\=1000 -e GID\=1000 -e USER\=mudge -e GROUP\=_staff -e GEM_PRIVATE_KEY_PASSPHRASE -e ftp_proxy -e http_proxy -e https_proxy -e RCD_HOST_RUBY_PLATFORM\=arm64-darwin22 -e RCD_HOST_RUBY_VERSION\=3.2.2 -e RCD_IMAGE\=ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux -w /Users/mudge/Projects/re2 --rm -i -t ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux runas sigfw bash -c gem\ install\ bundler\ --no-document\ \&\&'
'bundle\ \&\&'
'bundle\ exec\ rake\ gem:x86-linux:builder\ CMAKE\=/usr/local/bin/cmake'
']
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:89:in `block in exec'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:44:in `each'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:44:in `exec'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:14:in `sh'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock.rb:46:in `sh'
/Users/mudge/Projects/re2/Rakefile:75:in `block (3 levels) in <top (required)>'
Tasks: TOP => gem:native => gem:x86-linux
(See full trace by running task with --trace)

Here's the full detail from --trace, it looks like it's the "personality" error from above:

+ bundle exec rake gem:x86-linux --trace
** Invoke gem:x86-linux (first_time)
** Execute gem:x86-linux
rake-compiler-dock bash -c "gem install bundler --no-document &&\nbundle &&\nbundle exec rake gem:x86-linux:builder CMAKE=/usr/local/bin/cmake\n"
docker run -v /Users/mudge/Projects/re2:/Users/mudge/Projects/re2 -e UID\=1000 -e GID\=1000 -e USER\=mudge -e GROUP\=_staff -e GEM_PRIVATE_KEY_PASSPHRASE -e ftp_proxy -e http_proxy -e https_proxy -e RCD_HOST_RUBY_PLATFORM\=arm64-darwin22 -e RCD_HOST_RUBY_VERSION\=3.2.2 -e RCD_IMAGE\=ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux -w /Users/mudge/Projects/re2 --rm -i -t ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux runas sigfw bash -c gem\ install\ bundler\ --no-document\ \&\&'
'bundle\ \&\&'
'bundle\ exec\ rake\ gem:x86-linux:builder\ CMAKE\=/usr/local/bin/cmake'
'
WARNING: The requested image's platform (linux/386) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
linux32: Failed to set personality to linux32: Invalid argument
rake aborted!
Command failed with status (1): [docker run -v /Users/mudge/Projects/re2:/Users/mudge/Projects/re2 -e UID\=1000 -e GID\=1000 -e USER\=mudge -e GROUP\=_staff -e GEM_PRIVATE_KEY_PASSPHRASE -e ftp_proxy -e http_proxy -e https_proxy -e RCD_HOST_RUBY_PLATFORM\=arm64-darwin22 -e RCD_HOST_RUBY_VERSION\=3.2.2 -e RCD_IMAGE\=ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux -w /Users/mudge/Projects/re2 --rm -i -t ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux runas sigfw bash -c gem\ install\ bundler\ --no-document\ \&\&'
'bundle\ \&\&'
'bundle\ exec\ rake\ gem:x86-linux:builder\ CMAKE\=/usr/local/bin/cmake'
']
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:89:in `block in exec'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:44:in `each'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:44:in `exec'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock/starter.rb:14:in `sh'
/Users/mudge/.gem/ruby/3.2.2/gems/rake-compiler-dock-1.3.0/lib/rake_compiler_dock.rb:46:in `sh'
/Users/mudge/Projects/re2/Rakefile:75:in `block (3 levels) in <top (required)>'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/Users/mudge/.rubies/ruby-3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/Users/mudge/.rubies/ruby-3.2.2/bin/rake:25:in `load'
/Users/mudge/.rubies/ruby-3.2.2/bin/rake:25:in `<top (required)>'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli/exec.rb:58:in `load'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli/exec.rb:58:in `kernel_load'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli/exec.rb:23:in `run'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli.rb:492:in `exec'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli.rb:34:in `dispatch'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/cli.rb:28:in `start'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/exe/bundle:45:in `block in <top (required)>'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/Users/mudge/.gem/ruby/3.2.2/gems/bundler-2.4.12/exe/bundle:33:in `<top (required)>'
/Users/mudge/.gem/ruby/3.2.2/bin/bundle:25:in `load'
/Users/mudge/.gem/ruby/3.2.2/bin/bundle:25:in `<main>'

Answer 16 · 2023-08-11T13:41:58.000Z

@mudge I should have mentioned that it's best right now to build native gems on a Linux x86 machine to avoid the overhead of QEMU.

moby/moby#20634 (comment) suggests this linux32: Failed to set personality to linux32: Invalid argument message may be a seccomp issue. I'll have to poke around https://docs.docker.com/engine/security/seccomp/ to see what we can do with Docker for Mac.

Answer 17 · 2023-08-11T17:36:57.000Z

I can reproduce the problem via:

% docker run --platform linux/i386 -it ghcr.io/rake-compiler/rake-compiler-dock-image:1.3.0-mri-x86-linux

linux32: Failed to set personality to linux32: Invalid argument

I think this is just a limitation with Docker for Mac. I tried enabling the Rosetta emulation feature (docker/for-mac#6796 (comment)), but that didn't seem to help.

Answer 18 · 2023-08-12T09:14:30.000Z

In that case, perhaps it would be simplest to set up GitHub Actions to run the full cross-build and save all the artifacts with the default retention period?

For now, we can do it on every push but you could imagine doing this only on release and then I can manually download and publish the gems to RubyGems.

Answer 19 · 2023-08-15T20:10:04.000Z

@mudge The full cross-build artifacts were previously retained for 1 day, but I increased this to 7 days in #86.

You can see an example list of all the artifacts here: https://github.com/mudge/re2/actions/runs/5871088933

Answer 20 · 2023-08-15T20:14:07.000Z

Oh, but I just realized we may have to update the build script to use the tag instead the timestamp for the version.

Answer 21 · 2023-08-15T20:17:31.000Z

For now, can we hardcode the version to 2.0.0.beta1?

Answer 22 · 2023-09-13T17:45:42.000Z

I'm happy with the current setup: CI will build a gem with whatever is specified in version.rb and I can grab the resulting .gem files and push them to RubyGems.