ffi/ffi

1.14.2: missing compatible arch in ffi_c.bundle on M1 Mac mini with system Ruby

venkatasg opened this issue Β· 47 comments

I'm trying to build my Github Pages website locally on my M1 mac Mini using this guide. For now, Im trying to create a dummy GitHub Pages site following that guide command by command. However, whenever I execute bundle exec jekyll new --force --skip-bundle . or bundle exec jekyll serve, I get the following error:

~/S/jek  $ bundle exec jekyll serve
bundler: failed to load command: jekyll (/Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/bin/jekyll)
LoadError: dlopen(/Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi_c.bundle, 0x0009): missing compatible arch in /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi_c.bundle - /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi_c.bundle
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi.rb:6:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi.rb:6:in `rescue in <top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/ffi-1.14.2/lib/ffi.rb:3:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/sassc-2.4.0/lib/sassc/native.rb:3:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/sassc-2.4.0/lib/sassc/native.rb:3:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/sassc-2.4.0/lib/sassc.rb:31:in `require_relative'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/sassc-2.4.0/lib/sassc.rb:31:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll/converters/scss.rb:3:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll/converters/scss.rb:3:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll-sass-converter.rb:4:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll-sass-converter.rb:4:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-4.2.0/lib/jekyll.rb:195:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-4.2.0/lib/jekyll.rb:195:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-4.2.0/exe/jekyll:8:in `require'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/gems/jekyll-4.2.0/exe/jekyll:8:in `<top (required)>'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/bin/jekyll:23:in `load'
  /Users/venkat/Sites/jek/vendor/bundle/ruby/2.6.0/bin/jekyll:23:in `<top (required)>'

I'm using the system ruby and bundle installed on macOS Big Sur. I've tried a bunch of troubleshooting steps -- they all seem to point to an issue with ffi-1.14.2 loading the ffi_c.bundle. Any help would be appreciated.

Happy New Year!

I've seen many solutions. Some install ffi via brew in x86_64 mode, others in native. None have worked for me.

There's also the option of compiling yourself, although I don't know all the steps and don't think it helps.

I've seen many solutions. Some install ffi via brew in x86_64 mode, others in native. None have worked for me.

There's also the option of compiling yourself, although I don't know all the steps and don't think it helps.

Running in x86 (Rosetta) mode is not a solution and more a workaround. Native support would be much better.

Hmm so y'all can confirm there is a bug with ffi ? I was wondering if the bug might have been in jekyll.

Any idea if there's work on to fix it in the near future? I wish I could help, but I'm not familiar with this low level stuff at all.

@venkatasg Have to correct myself, the issue seems to be the version jekyll is depending on. Jekyll still depends on 1.13 while the support for M1 seems to have been implemented in 1.14.
Unfortunately it seems the fix is not working as I am still getting the error with the newest version of jekyll which uses 1.14.

For now the only solution seems to be the rosetta workaround:

Install: arch -arch x86_64 bundle install
Start: arch -arch x86_64 bundle exec jekyll serve

Thank you so much! Should we open an issue on the jekyll repo then instead of here?

You're welcome! Both projects need to update, but as long as the problem resides in ffi on M1 I think it makes no sense for jekyll to update its dependencies.

Fwiw, native ffi 1.14.2 works fine for me, but I'm using a native-only Ruby build (via rbenv), not the system-installed Ruby (which is a universal binary).

After playing around a bit I found out that the pre-installed ruby on macOS seems to falsely recognize the CPU as x86, no matter what you tell it to do:

~ lipo -archs /usr/bin/ruby        
x86_64 arm64e
~ ruby -v                          
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.arm64e-darwin20]
~ arch -c -arch arm64e ruby test.rb
OS: darwin20
CPU x86_64
Arch: universal-darwin20

So the problems seems to be more deeply than we thought and we might just have to wait for Apple to update the pre-installed ruby.

I guess we need to file a Feedback/Radar with Apple then? I might get around to it sometime, but I've never gotten any response on any of the feedbacks I've filed thus far, so I've learnt never to place high trust in that.

Should I close this issue?

I've seen many solutions. Some install ffi via brew in x86_64 mode, others in native. None have worked for me.
There's also the option of compiling yourself, although I don't know all the steps and don't think it helps.

Running in x86 (Rosetta) mode is not a solution and more a workaround. Native support would be much better.

What's so frustrating is some say that everything works fine, but don't say whether they used the solution and/or workaround they used. So it's been very frustrating. I was hoping for a constructive reply, but dismayed that even though I clearly said "in x86_64 mode" you'd have some subjective semantical objection to a word in the previous sentence. Anyway, it's hard to tell if anyone has it working natively or not.

After playing around a bit I found out that the pre-installed ruby on macOS seems to falsely recognize the CPU as x86, no matter what you tell it to do:

This might explain reports of only using Rosetta to install ffi as a fix for some projects/frameworks. But I think those have all been early victories, that lead to folks needing to use Rosetta for everything else too. Could using another installation of Ruby be a better option for now?

This might explain reports of only using Rosetta to install ffi as a fix for some projects/frameworks. But I think those have all been early victories, that lead to folks needing to use Rosetta for everything else too. Could using another installation of Ruby be a better option for now?

It seems to be the only "solution" for now and it might be an explanation why some people are reporting they have no problems and / or run ruby natively. I just think some people might not have the permissions to install custom ruby installations, so this is no option for them.

Nevertheless it's still a bug, that the ruby universal binary is not choosing the correct architecture which needs to be fixed by Apple.

Nevertheless it's still a bug, that the ruby universal binary is not choosing the correct architecture which needs to be fixed by Apple.

Look, I want to share your discovery above in other threads on the topics. And I'd love to include what you or this community suggest is the best resolution until one comes from Apple. I know when I make posts, I sometimes feel they're glossed over and it's annoying. Then I've made responses that try redirect attention to my post. So, sorry if I didn't praise your discovery at the time. But I have the link in my clipboard to show others, I was just asking you and company that since we now know more or less where the problem lies, what's the best way to keep working while we wait for Apple's vaccine? While I hope most @sanzaru has a temporary method, I ask you you only reply with it if you do, and not to reply at all if it's to tell me only how you would have worded something or other different. I'm just finding that irritating right now (if not always).

Open to everyone's thoughts that may help folks get through the wait. Thank you.

Fwiw, native ffi 1.14.2 works fine for me, but I'm using a native-only Ruby build (via rbenv), not the system-installed Ruby (which is a universal binary).

What version did you install?

What version did you install?

$ rbenv which ruby
/Users/kameliya/.rbenv/versions/2.7.2/bin/ruby
$ ruby -v
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [arm64-darwin20]

Apple added deprecation notices to the system Ruby installation starting in 10.15 (Catalina). Fire up irb and you'll see this:

WARNING: This version of ruby is included in macOS for compatibility with legacy software.
In future versions of macOS the ruby runtime will not be available by
default, and may require you to install an additional package.

I'm not entirely sure how users are supposed to know about this though, I just recall it being reported at the time. Might as well be on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying β€œBeware of the Leopard.”

@venkatasg you're "using the system ruby and bundle" which goes against this recommendation and that of the article, which tells you to go to the Ruby site for installation options, not use the preinstalled one. Again I'm not faulting you here, it's a frustrating/confusing situation for sure when these utilities are "available" but often ancient and/or broken.

It might be a good idea to update the title of this bug to make abundantly clear we're talking about the system Ruby installation. (update: πŸ‘πŸ»)

@sanzaru could you provide the script you used? I can help file a radar and share the appropriate Open Radar here if no one is doing that.

Apple added deprecation notices to the system Ruby installation starting in 10.15 (Catalina). Fire up irb and you'll see this:

WARNING: This version of ruby is included in macOS for compatibility with legacy software.
In future versions of macOS the ruby runtime will not be available by
default, and may require you to install an additional package.

I'm not entirely sure how users are supposed to know about this though, I just recall it being reported at the time. Might as well be on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying β€œBeware of the Leopard.”

That's interesting. I didn't know about this warning at all. Could be more transparent for devs...

@sanzaru could you provide the script you used? I can help file a radar and share the appropriate Open Radar here if no one is doing that.

Of course. It was quite simple:

require 'rbconfig'

OSVERSION = RbConfig::CONFIG['host_os']
CPU = RbConfig::CONFIG['host_cpu']
ARCH = RbConfig::CONFIG['arch']

puts "OS: #{OSVERSION}"
puts "CPU #{CPU}"
puts "Arch: #{ARCH}"

I found out that the RbConfig module (rbconfig.rb) seems to be the problem as it was compiled for x86 and therefore always returns that value when reading the module. Hope I can publish a detailed article about that, soon.

Of course. It was quite simple:

require 'rbconfig'

OSVERSION = RbConfig::CONFIG['host_os']
CPU = RbConfig::CONFIG['host_cpu']
ARCH = RbConfig::CONFIG['arch']

puts "OS: #{OSVERSION}"
puts "CPU #{CPU}"
puts "Arch: #{ARCH}"

Sorry, do you need to put this file somewhere and then compile Ruby?

You can put it in a file (e.g. call it arch.rb) and then run ruby arch.rb to get the results.

Is there any obvious fix for RbConfig being wrong on native Ruby on M1?

Is there any obvious fix for RbConfig being wrong on native Ruby on M1?

Not that I know of. The only "solution" is to install your own ruby (e.g. via Homebrew) and use that. The binary from Apple is deprecated. I wrote an article about my findings, if you're interested: https://medium.com/better-programming/ruby-on-apple-silicon-m1-macs-fb159849b2f5

We found a potential solution here: socketry/nio4r#267

@venkatasg Have to correct myself, the issue seems to be the version jekyll is depending on. Jekyll still depends on 1.13 while the support for M1 seems to have been implemented in 1.14.
Unfortunately it seems the fix is not working as I am still getting the error with the newest version of jekyll which uses 1.14.

For now the only solution seems to be the rosetta workaround:

Install: arch -arch x86_64 bundle install
Start: arch -arch x86_64 bundle exec jekyll serve

Sorry but where do I run 'arch -arch x86_64 bundle install`? It says no gem file found. Please help!

@venkatasg Have to correct myself, the issue seems to be the version jekyll is depending on. Jekyll still depends on 1.13 while the support for M1 seems to have been implemented in 1.14.
Unfortunately it seems the fix is not working as I am still getting the error with the newest version of jekyll which uses 1.14.
For now the only solution seems to be the rosetta workaround:
Install: arch -arch x86_64 bundle install
Start: arch -arch x86_64 bundle exec jekyll serve

Sorry but where do I run 'arch -arch x86_64 bundle install`? It says no gem file found. Please help!

@StupidRepo Make sure to run the command inside a folder that contains a Gemfile.

@venkatasg Have to correct myself, the issue seems to be the version jekyll is depending on. Jekyll still depends on 1.13 while the support for M1 seems to have been implemented in 1.14.
Unfortunately it seems the fix is not working as I am still getting the error with the newest version of jekyll which uses 1.14.
For now the only solution seems to be the rosetta workaround:
Install: arch -arch x86_64 bundle install
Start: arch -arch x86_64 bundle exec jekyll serve

Sorry but where do I run 'arch -arch x86_64 bundle install`? It says no gem file found. Please help!

@StupidRepo Make sure to run the command inside a folder that contains a Gemfile.

Thank you but when setting it up I get An error occurred while installing http_parser.rb (0.6.0), and Bundler cannot continue. but no matter what I do I can't fix it! I use the arch command but it's still not working. Sorry for hijacking this thread

I did run the command it told me to run (gem install http_parser.rb -v '0.6.0' --source 'https://rubygems.org/') but that said it installed but still gives the same error.

Building native extensions. This could take a while...
Successfully installed http_parser.rb-0.6.0
Parsing documentation for http_parser.rb-0.6.0
unknown encoding name "chunked\r\n\r\n25" for ext/ruby_http_parser/vendor/http-parser-java/tools/parse_tests.rb, skipping
Done installing documentation for http_parser.rb after 0 seconds
1 gem installed

I've seen many solutions. Some install ffi via brew in x86_64 mode, others in native. None have worked for me.

It seems one solution is to run whole terminal in Rosetta mode (go to apps -> tools -> terminal -> right click -> get info -> open using rosetta) and then install everything in that mode (I use XCode in rosetta and that works for my mobile dev env) - but personally I gave up on that idea - you have to juggle with arch and it's becomes a pain quickly - and simply started running M1-incompatbile projects in docker container via docker compose - there is already m1 tech previews available and it's pretty stable; everything else - I try to install natively - if it breaks - docker to the rescue; my 2-cents-workaround 🀷

I've seen many solutions. Some install ffi via brew in x86_64 mode, others in native. None have worked for me.

It seems one solution is to run whole terminal in Rosetta mode (go to apps -> tools -> terminal -> right click -> get info -> open using rosetta) and then install everything in that mode (I use XCode in rosetta and that works for my mobile dev env) - but personally I gave up on that idea - you have to juggle with arch and it's becomes a pain quickly - and simply started running M1-incompatbile projects in docker container via docker compose - there is already m1 tech previews available and it's pretty stable; everything else - I try to install natively - if it breaks - docker to the rescue; my 2-cents-workaround 🀷

Nope, still doesn't work for me when in Rosetta

Any status update on that?
We cannot setup the CI on M1 Mac because of this issue.

@mkocus: At least as of 2021-06-21: if you make sure to install ruby from homebrew, and make sure PATH points to this newer version of ruby (rather than the binary installed by default on macOS), installing packages that make use of ffi should work just fine. (At least I have not run into any problems).

@venkatasg Have to correct myself, the issue seems to be the version jekyll is depending on. Jekyll still depends on 1.13 while the support for M1 seems to have been implemented in 1.14.
Unfortunately it seems the fix is not working as I am still getting the error with the newest version of jekyll which uses 1.14.

For now the only solution seems to be the rosetta workaround:

Install: arch -arch x86_64 bundle install
Start: arch -arch x86_64 bundle exec jekyll serve

I go this when I ran the command

arch: posix_spawnp: bundle: Bad CPU type in executable

I go this when I ran the command

arch: posix_spawnp: bundle: Bad CPU type in executable

@Basitomania Did you install Rosetta? This error normally occurs when Rosetta is missing.

Try: softwareupdate --install-rosetta

For me, it seems that I got 2 different versions of ffi and it was choosing the bad one so first uninstall both:
sudo gem uninstall ffi
Then proceed with the above.

I upgraded the Ruby to 3.0.2 with asdf as you can follow steps here.

After that I needed to update bundle,

bundle update
bundle add webrick

you can also follow steps described here.

As a result of that bundle exec jekyll serve works fine on M1 mac

I've been working around this by using devcontainer in vscode: https://github.com/DelfinaCare/jekyll-devcontainer or microsoft/vscode-dev-containers#1038 as examples

from an Above comment :

I upgraded the Ruby to 3.0.2 with asdf as you can follow steps here.

After that I needed to update bundle,

bundle update
bundle add webrick

you can also follow steps described here.

As a result of that bundle exec jekyll serve works fine on M1 mac

This worked for me! here's how...

I'm using an Apple M1 Mac Mini , running Big Sur 11.6

some notes on the solution:

It was important for me to realize I was using zsh now, instead of Bash like I was used to on my old Mac laptop. This is a new default, so I installed Oh My Zsh to make things easier going forward for myself. ymmv.

Homebrew installation of ruby did me dirty because there was a command that is supposed to update the $PATH, but it wasn't working.

This was due to homebrew some how, I forget now sorry I should have noted it... but anyways, I uninstalled the home-brew version of ruby.

No matter what with this home-brew version, when I typed ruby -v , it would tell me that I am using the 2.x.x version not the 3.x.x. This is the Mac OS default ruby version.

So I installed a fresh ruby after deleting the home-brew ruby.
which ruby
where ruby
helped me here. It went from being in the something/homebrew/something/ruby location to the usr / myname / ruby location. Fresh install confirmed.

The next hurdle was how to tell the system to use this 3.x x version of ruby by default and not the 2.x.x oldie one.

For this, I installed a 'manager' called 'asdf', which allowed me to fix the issues with my $PATH. After following the asdf install instructions, what worked in the end was running asdf reshim ruby.

Then when I typed ruby -v it showed the correctt 3.x.x version. Hurray.

Then, with ruby working, I moved on to jekyll.

I followed this advice to uninstall my accidental jekyll installs just to start fresh to not mess myself up somehow. turns out I had a few extra installs so I nuked them all and followed the generic instructions to get my fresh install. I thought everything was done right but then I encountered this thread's main topic, the issue with the m1 chip.

So after having installed jekyll,
then Jekyll new mysitename ,
then cd mysitename

then

bundle update
bundle add webrick

then

bundle exec jekyll serve WORKED!!!!

Thank you :)

Got it to work by using system ffi :

brew install libffi
gem uninstall ffi
gem install ffi -- --enable-system-libffi

gem install ffi -- --enable-system-libffi

When I run the command gem install ffi -- --enable-system-libffi I get this error system libffi is not usable

I had an existing Jekyll project that hit the ffi issue. It was already placing all code into vendor/bundle

I was able to install a new libffi with brew, setup things to use it rather than the system version, and then the bundle install succeeded (which included the ffi gem).

brew install automake libffi pkg-config
...
export LDFLAGS="-L/opt/homebrew/opt/libffi/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libffi/include"
bundle install

Got it to work by using system ffi :

brew install libffi
gem uninstall ffi
gem install ffi -- --enable-system-libffi

pod install worked after I ran this. Thanks @charlesvallieres

Was running similar problems with Flutter and cocoapods. From here, https://github.com/flutter/flutter/wiki/Developing-with-Flutter-on-Apple-Silicon the following worked for me:

sudo gem uninstall ffi && sudo gem install ffi -- --enable-libffi-alloc

Was running similar problems with Flutter and cocoapods. From here, https://github.com/flutter/flutter/wiki/Developing-with-Flutter-on-Apple-Silicon the following worked for me:

sudo gem uninstall ffi && sudo gem install ffi -- --enable-libffi-alloc

This did the trick for me! Thanks Stef!

On my M1, I enabled rosetta on xcode and terminal, to support older projects.
Then had to do this:

arch -arm64 brew reinstall libffi
gem uninstall ffi
arch -arm64 gem install ffi

PS: I'm using cocoapods from gem and not from brew.

@sanzaru Perhaps Apple preinstalled ruby has been updated to use the correct CPU Instruction Set Architecture (ISA) slice from the universal binary?

After playing around a bit I found out that the pre-installed ruby on macOS seems to falsely recognize the CPU as x86, no matter what you tell it to do:

~ ruby -v                          
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.arm64e-darwin20]
~ arch -c -arch arm64e ruby test.rb
OS: darwin20
CPU x86_64
Arch: universal-darwin20

Below are some results from macOS Ventura 13.4 (22F66).

which -a ruby
# /usr/bin/ruby

ruby -v
# ruby 2.6.10p210 (2022-04-12 revision 67958) [universal.arm64e-darwin22]

ruby test.rb
# OS: darwin22
# CPU aarch64
# Arch: universal-darwin22

arch -x86_64 ruby test.rb
# OS: darwin22
# CPU x86_64
# Arch: universal-darwin22

arch -arm64 ruby test.rb
# OS: darwin22
# CPU aarch64
# Arch: universal-darwin22

arch -arm64e ruby test.rb
# OS: darwin22
# CPU aarch64
# Arch: universal-darwin22

arch -c -arch x86_64 ruby test.rb
# OS: darwin22
# CPU x86_64
# Arch: universal-darwin22

arch -c -arch arm64 ruby test.rb
# OS: darwin22
# CPU aarch64
# Arch: universal-darwin22

arch -c -arch arm64e ruby test.rb
# OS: darwin22
# CPU aarch64
# Arch: universal-darwin22

On my M1, I enabled rosetta on xcode and terminal, to support older projects. Then had to do this:

arch -arm64 brew reinstall libffi
gem uninstall ffi
arch -arm64 gem install ffi

PS: I'm using cocoapods from gem and not from brew.

This worked for me on an M2 mac pro, I am not using rosetta

I had these issue a few minutes ago. Sound weird, but the solution was to start Xcode. I updated Xcode a few days ago and the licence had to be acknoledged, after that I could run bundle install and ffi was installed without any issues.