Apple Silicon plan
mcollina opened this issue ยท 63 comments
I'm opening a tracking issue for our plan regarding apple silicon, including our dependencies.
It might be possible to source some development kits via the Foundation if we need them, as this was done in the past for our CI. Should we work in that direction?
We have a few high meta questions to answer: should we look to backport the support to 14.x or 12.x? Or are we targeting 15.x only?
I don't think there is a lot of time.
(If any person of Apple wants to join the discussion as you said you would this is a good spot :)).
I wouldn't imagine we need to make too many changes, they're just arm right?
This definitely should be connected to the Windows ARM64 support as well. We have support in for that, so I wouldn't expect there to be much left to do to support Apple Silicon but it's definitely something we need to audit and make sure we're fully supporting.
I'm not sure what is the problem we are trying to solve here.
For example, what are the potential problems and what might be needed to be done, except for extending the CI matrix?
Upd: ah, @jasnell comment from seconds ago explains it.
Specifically, among other things we need to verify...
- Do all of our native dependencies support ARM64
- brotli
- cares
- histogram
- icu-small
- llhttp
- nghttp2
- nghttp3
- ngtcp2
- openssl
- uv
- uvwasi
- v8
- zlib
- Do all of our build tools support ARM64
- python
- gyp
- c/c++ linter
- cctest
- icu
- code-cache
- snapshot
- Do we have CI coverage
- Apple Silicon
- Windows ARM64
The answer is likely yes to all most of these and we can likely check everything off but we should make sure.
I'm not sure what is the problem we are trying to solve here.
Logistics.
My current issue as the CPC representative is to check if we need to try to buy a few dev kits and give them to collaborators and/or add them to our CI. Without the devices, we can't do much.
There's probably some non-trivial work to do to teach GYP to generate ARM projects for XCode
Without test machines for the CI any support would be experimental at best.
@jasnell We'd also need to verify python and gyp in terms of build tools.
We have a few high meta questions to answer: should we look to backport the support to 14.x or 12.x? Or are we targeting 15.x only?
For backporting I think that really depends on what (if any) changes are required. I'm particularly concerned if we'll need to introduce branches in the gyp files to support ARM on macOS.
@jasnell We'd also need to verify python and gyp in terms of build tools.
Added to the checklist
My current issue as the CPC representative is to check if we need to try to buy a few dev kits and give them to collaborators and/or add them to our CI. Without the devices, we can't do much.
As part of this, let's explore also adding Windows ARM64 to the CI test matrix.
@AshCripps can you check with our OSX infra donator to see if they have any plans to include ARM64 support.
(If any person of Apple wants to join the discussion as you said you would this is a good spot :)).
If this tweet I RT'd is legit (possibly from WWDC - I wasn't watching!) then they may already be looking at it: https://twitter.com/MarkVillacampa/status/1275200446764912643
My current issue as the CPC representative is to check if we need to try to buy a few dev kits and give them to collaborators and/or add them to our CI. Without the devices, we can't do much.
As part of this, let's explore also adding Windows ARM64 to the CI test matrix.
https://ci.nodejs.org/job/node-test-commit-windows-arm64/
History:
@richardlau ... yeah, I know that job has existed but as far as I know and I could be mistaken) it is not run as part of regular CI runs, correct?
@richardlau ... yeah, I know that job has existed but as far as I know and I could be mistaken) it is not run as part of regular CI runs, correct?
Correct. I think we even stopped producing unofficial builds for Windows ARM (AFAIK).
...if we need to try to buy a few dev kits and give them to collaborators and/or add them to our CI. Without the devices, we can't do much.
@mcollina ... likely a good idea, with priority going to members of the @nodejs/build team so that they can work out any possible issues getting things to compile and run.
Message from macstadium with regards to supporting apple sillicon in ORKA (our current "modern" macos hosting solution)
Hi @Ash Cripps. Weโve got some DTKs heading our way. Until we get hands-on itโs hard to say, but weโre hopeful. Weโll announce our progress as we go.
So they plan to try support it but no guarantees yet as its still early days
Itโd be great if someone could test nvm on the new hardware and let me know if thereโs anything that needs fixing.
I've submitted a request for a developer transition kit to Apple via the Node.js Apple Developer account. I also included a link back to this issue in the notes.
Thank you for your interest in the Universal App Quick Start Program. Weโll review your application and get back to you soon.
Since we've had extensive testing on ARM64 for Linux for quite a while now and some limited testing for Windows now I don't imagine there'll be major problems on the code and dependency side of things. It's all the toolchain pieces I'm more concerned about.
Re Windows ARM64: the problem is hardware availability. We've been relying on laptops on @joaocgreis' desk to do the testing and unofficial builds. Because it's mainly designed for use on laptops there's not really many other options just yet.
I've submitted a request for a developer transition kit to Apple via the Node.js Apple Developer account. I also included a link back to this issue in the notes.
Awesome. Do you need me to do anything foundation-wise?
Will you order that kit and host it with the pi in your basement?
Awesome. Do you need me to do anything foundation-wise?
Will you order that kit and host it with the pi in your basement?
For now we have to wait for review now from Apple. If they accept that we're a good "investment" they'll send an invoice and arrange delivery. I can get the Foundation to pay for it (Brian has a billing account in the team we have in our Apple Developer account and I think can pay from thereโperhaps you need to authorise it on behalf of the TSC?). I'll keep this thread updated with any progress (it's going through build@iojs.org so Michael and others can see any progress too).
I could host it with the Pi cluster, since it's likely got a limited lifetime that's probably not a big deal. It could go to nearForm with our other Macs but I suspect this is going to need much more custom handling and fiddling than a standard MacMini. Apple do have language about this being "Apple owned" so it might be something we have to ship back to them at a certain date.
Hopefully MacStadium comes through with Orka but I'm betting that's going to be a massive investment for them and might take time. It'd probably be wise of us to plan on investing in new hardware sometime in 2021 to support this to parallel the MacMinis we have at nearForm as a backup. Hopefully they have an ARM MacMini out alongside their new Macbooks.
@ljharb FWIW we have am ARM DTK machine and trying to use nvm gives the following error:
Downloading and installing node v12.6.0...
Downloading https://nodejs.org/dist/v12.6.0/node-v12.6.0-darwin-arm64.tar.gz...
#=#=#
curl: (22) The requested URL returned error: 404
Binary download from https://nodejs.org/dist/v12.6.0/node-v12.6.0-darwin-arm64.tar.gz failed, trying source.
@jedrichards well we don't have official darwin-arm64 releases, if we did nvm would work fine :) If you're in the mood for more testing, trying to build node from source would be useful.
The last quoted line from the nvm log said "... trying source.". Did it, and if so what happened?
That's as much info as I have at the mo (I don't personally have the DTK). I'll ask for more info, and request that they try building from source too.
More context to @jedrichards' comment:
it turns out that Node.js 12.6.0 fails building on the DTK:
In file included from: ../deps/v8/src/base/platform/mutex.h:9:
In file included from: ../deps/v8/src/base/lazy-instance:73:
In file included from: ../deps/v8/src/base/macros.h:11:
In file included from: ../deps/v8/src/base/format-macro.h:27:
#error Target architecture arm64 is only supported on arm64 and x64host
In file included from: ../deps/v8/src/libplatform/delayed-task-queue.cc:5:
In file included from: ../deps/v8/src/libplatform/delayed-task-queue.h:12:
In file included from: ../deps/v8/src/base/macros.h:11:
In file included from: ../deps/v8/src/base/format-macro.h:27:
#error Target architecture arm64 is only supported on arm64 and x64host
In file included from: ../deps/v8/src/libplatform/task-queue.cc:5:
In file included from: ../deps/v8/src/libplatform/task-queue.h:11:
In file included from: ../deps/v8/src/base/macros.h:11:
In file included from: ../deps/v8/src/base/format-macro.h:27:
#error Target architecture arm64 is only supported on arm64 and x64hostAfter changing the Node.js version in .nvmrc to 12.18.2 and running it again, it succeeded building ๐
@jedrichards nvm automatically installs from source if the binary fails, so that's the output i'm interested in :-) thanks!
I just tried (nvm 0.35.3, nvm i 14, on the DTK) and it did fall back to downloading the source & attempting to compile, but ends up failing with a bunch of errors in ../deps/openssl/config/archs/linux-x86_64/asm_avx2/crypto/aes/ (which I guess might be related to openssl/openssl#12254)
Full dump from my terminal if it helps.
Update: Macstadium are very generously giving us access to 2 DTKs for no cost! ๐ we have a ticket open so will update when we have access.
ive opened a PR in build to add the DTKs to our CI - nodejs/build#2383.
I plan to create a new job for just the DTKs so people are able to test PRs etc against apple sillicon and Big Sur.
Would we want this job to eventually be added to node-test-commit as well?
@AshCripps I think so provided the machines we have are fast enough.
https://ci.nodejs.org/job/node-test-commit-osx-arm/ we now have a job to test on these macs, works the same as other node-test-commit jobs. Any issues LMK.
This is pretty amazing of MacStadium, they deserve lots of love for this, can we get some @nodejs tweets or other media out there to thank them as a key part of our journey to macOS ARM? Maybe some @nodejs/tsc members with larger followings could do some public thanking.
This is pretty amazing of MacStadium, they deserve lots of love for this, can we get some
@nodejstweets or other media out there to thank them as a key part of our journey to macOS ARM? Maybe some @nodejs/tsc members with larger followings could do some public thanking.
/ping @nodejs/social
So once 34238 is landed, will we add the osx-arm runner to node-test-pull-request?
My DTK arrived today and I'd like to help where ever I can docs, builds, testing. I need a little direction on what's needed and where to start and I'll do what I can.
@jtomchak one thing that was discussed in the build WG, would be that you could help with respect to running the benchmarks to help identify where there might be performance issues.
Is there any update on this, please? Maybe a linked nightly build for macOS/Windows ARMs? ๐
@jakubzitny the support should be landed on master and we have some test machine in ci - I just to add them to the regular regression test runs (Will do that today its slipped my mind). ATM I dont think we have plans to produce the binaries and we havent decided on what we are doing IIRC.
I think we should start planning official releases for Apple Silicon.
I agree Ill open a build issue to kickstart the discussions again - I think we should be ok on launch due to the rosetta stone translation layer but that wont be there forever
I am a front-end developer. Is it suitable to use Apple M1 ARM chip?
@FuncWei it depends on what you're doing exactly, but to me, it's not totally ready yet. I have problems during yarn install of my node 12.x project. Also Docker is not working yet
I came across this handy site that shows the current status of a number of apps, languages, developer tools, etc. Might help other people that come across this issue.
@FuncWei for me everything I need works great. Btw I don't need docker.
I use nvm and node v15.3.0 build from source.
I use nvm and node v15.3.0 build from source.
In my case, using nvm to build v15.3.0 from source still produced an x86 binary which would run via rosetta. (ref: nodejs/build#2474 (comment))
You can verify using: node -p process.arch -> it should print arm64 (using nvm install v15 it still printed out x64).
I was able to get a correct arm build using homebrew, however.
Note that homebrew is not yet officially supported on Apple Sillicon so you need to install it using the alternative method to /opt/homebrew:
cd /opt
sudo mkdir homebrew
sudo chown -R $(whoami) homebrew
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew
cd /opt/homebrew/bin
./brew install -s node(also helpful: https://stackoverflow.com/questions/64951024/how-can-i-run-two-isolated-installations-of-homebrew/64951025#64951025)
It should start building, and it will take around 20 minutes.
Now if you run node -p process.arch it correctly prints out arm64 (note that the real location of the node binary will be in /opt/homebrew/opt/node/bin/)
I use nvm and node v15.3.0 build from source.
In my case, using nvm to build v15.3.0 from source still produced an x86 binary which would run via rosetta. (ref: nodejs/build#2474 (comment))
I used nvm to install 15.3.0 and it gave me arm64. You might have somehow run your shell (bash or zsh) or terminal app in Rosetta before.
I used nvm to install 15.3.0 and it gave me arm64. You might have somehow run your shell (bash or zsh) or terminal app in Rosetta before.
Running arch in Terminal prints arm64 - so the terminal is running on the correct architecture.
I tried uninstalling and reinstalling nvm and letting it build node several times and always ended up with an x64 binary. Weird.
ๆ้ๅฐไบ่ฟไธชๆนไพฟ็็ซ็น๏ผ่ฏฅ็ซ็นๆพ็คบไบ่ฎธๅคๅบ็จ็จๅบ๏ผ่ฏญ่จ๏ผๅผๅไบบๅๅทฅๅ ท็ญ็ๅฝๅ็ถๆใๅฏ่ฝไผๅธฎๅฉ้ๅฐๆญค้ฎ้ข็ๅ ถไปไบบใ
Yes, and this website https://doesitarm.com/
I use nvm and node v15.3.0 build from source.
In my case, using nvm to build v15.3.0 from source still produced an x86 binary which would run via rosetta. (ref: nodejs/build#2474 (comment))
I used
nvmto install 15.3.0 and it gave mearm64. You might have somehow run your shell (bash or zsh) or terminal app in Rosetta before.
Anyone have step-by-step instructions?
@playvici: #886 (comment)
I'd suggest locking this thread to collaborators and opening a separate issue over on https://github.com/nodejs/help/ for all the people that are just debugging their individual setups
@playvici
Here https://ported2m1.com/app/kcnaqU7k7CMDVmQWuvPa
I was able to install it through brew.
@playvici
Here https://ported2m1.com/app/kcnaqU7k7CMDVmQWuvPa
I was able to install it through brew.
do i need to uninstall x86 brew and reinstall as arm?
I really need node v12 in mac silicon
@playvici
Here https://ported2m1.com/app/kcnaqU7k7CMDVmQWuvPa
I was able to install it through brew.do i need to uninstall x86 brew and reinstall as arm?
@playvici it depends on you. I keep both the versions.
I have added an alias for it, brewin
So, I use > brewin install... for x86 binaries
And > brew install... for M1 binaries
Add this to end of your zshrc/bashrc file after installing both versions.
# Brew Intel
alias brewin="arch -x86_64 /usr/local/bin/brew"
# Brew native
export PATH=/opt/homebrew/bin:$PATH
You can try installing something with brew install (for native M1), if that fails install it with brewin (Intel using Rosetta).
To install Node 15.x arm64 on M1, you need:
- Install latest version of homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" brew install node- check version by
node -p process.archthis should printarm64
As per #886 (comment) I went ahead and locked the issue to collaborators. Please open new issues in nodejs/help to discuss debugging individual setups.
@nodejs/build It would be good to have an update on the progress of getting an official build out. I could not find any easy-to-follow tracking issue.
@nodejs/tsc as we can see, this is a highly requested feature. I think we might want to flag it as a "Strategic Initialive".
V16 shipped with M1 support so closing :)